Probability-based Online Panel Research Articles

Erectile dysfunction prevalence in the United States: report from the 2021 National Survey of Sexual Wellbeing.

There has been a great deal of public speculation regarding a surge in erectile dysfunction (ED) in younger men despite data consistently indicating that the ED prevalence rates increase with age. In this study we sought to assess the prevalence and risk of experiencing ED in a nationally representative sample of men in the United States across various social groups, describe comorbidities, and examine barriers to treatment. Data from the 2021 National Survey of Sexual Wellbeing were analyzed by utilizing the Ipsos KnowledgePanel®, a probability-based online panel, for the purpose of obtaining US nationally representative data of adults aged 18 years and older. The analytic sample consisted of 1822 cisgender men ranging in age from 18 to 87 years, with a mean age of 47.5 years. Study outcomes were ED as measured by the 5-item version of the International Index of Erectile Function (IIEF-5), as well as self-reported diagnosis by a medical professional, comorbidities with other health issues, medications taken for ED, and barriers to treatment. The ED prevalence rate based on IIEF-5 scores was 24.2%. Prevalence increased with age: 52.2% of the 75+ age group, and 48.0% of the 65-74 age group meeting diagnostic criteria for ED. Diagnostic criteria were met for more participants in the 18-24 age group (17.9%) than the 25-34 (13.3%) or 35-44 (12.7%) age groups, but less than the 45-54 (25.3%) or 55-64 (33.9%) age groups. Only 7.7% (n = 141) of the sample reported having been diagnosed by a provider (n = 4 in 25-34, n = 6 in 35-44, n = 13 in 45-54, n = 39 in 55-64, n = 44 in 65-74, and n = 34 in ≥75-year olds), indicating a gap in access to treatment. The most common reason selected for not accessing care for ED concerns was a lack of need to see a provider in the past year. The discrepancy between ED self-report and medical diagnosis is critical, given that ED can be an indication of underlying health risks. This study is the first nationally representative update to information regarding the prevalence of ED in almost 2 decades. Standard administration of the IIEF-5 is limited to participants who have had sex in the last 6 months, so these rates do not include those men with ED who have been avoiding penetrative sex for a significant time. The results indicate that ED prevalence and severity remain highest in older age groups and that most individuals who meet criteria for ED have not sought medical care related to this concern.

Contexts and Characteristics of Imaged-Based Sexual Exploitation and Abuse of Children: Incident Dynamics in a National Sample.

This paper describes incident characteristics and dynamics associated with six specific forms of image-based sexual exploitation and abuse of children (IBSEAC). Data were collected on a national sample of 2639 individuals aged 18-to-28 from a probability-based online panel. Respondents completed a self-administered survey questionnaire, providing detailed follow-up information on their experiences of image-based sexual victimization before the age of 18. A total of 607 incidents of IBSEAC were included in the analyses. Findings show substantial diversity in incident characteristics within and across the six forms of IBSEAC (nonconsensual sharing of sexual images, nonconsensual taking or making of images, forced image recruitment, threatened sharing, voluntary image sharing with an older adult, and commercial sexual exploitation involving images). Some notable patterns include frequent involvement of perpetrators who are other youth or young adults, who are known in-person to the victim, and who are intimate partners. The diversity and complexity of dynamics revealed in this study underscores the need for careful design and evaluation of prevention programs and the core messages directed at youth.

Investigating Respondent Attention to Experimental Text Lengths

Abstract Whether respondents pay adequate attention to a questionnaire has long been of concern to survey researchers. In this study, we measure respondents’ attention with an instruction manipulation check. We investigate which respondents read question texts of experimentally varied lengths and which become inattentive in a probability-based online panel of the German population. We find that respondent attention is closely linked to text length. Individual response speed is strongly correlated with respondent attention, but a fixed cutoff time is unsuitable as a standalone attention indicator. Differing levels of attention are also associated with respondents’ age, gender, education, panel experience, and the device used to complete the survey. Removal of inattentive respondents is thus likely to result in a biased remaining sample. Instead, questions should be curtailed to encourage respondents of different backgrounds and abilities to read them attentively and provide optimized answers.

A meta-analysis of worldwide recruitment rates in 23 probability-based online panels, between 2007 and 2019

ABSTRACT This study is a meta-analysis of overall recruitment rate (ORR) in probability-based online panel research. In this study, we included 23 general population probability-based online panels (out of 28 identified and described) covering 15 countries and provided a comprehensive overview of their methodological approaches to recruitment. We calculated the average ORR and carried out a meta-regression with selected moderators. On average less than 1 out of 5 eligible sampled people registered as panel members, but the data showed a very high heterogeneity of ORRs. We found that recruitment rates were similar in telephone, face-to-face, and mail modes in the United States (U.S.), but the mail mode performed worse than the interviewer-administered recruitment modes in the rest of the world. Also, we identified a small decrease of ORR over time and provided evidence on how ORRs are significantly lower in the U.S. probability-based online panels.

Physical activity and mental health: Comparing between-person and within-person associations in longitudinal analysis

BackgroundExtensive research has consistently highlighted the strong association between physical activity and mental health. But few have examined how this association may differ at the between-person and the within-person level. This longitudinal study compares how within-person and between-person changes in physical activity are associated mental health during the COVID-19 pandemic among the American population. MethodsWe analyzed data from the Understanding Coronavirus in America (UCA) survey, a probability-based online panel of American adults. An average of more than 6000 panel members repeatedly participated across 33 waves from the start of the pandemic in March 2020 to January 2023. Measures of physical activity include 1) number of days exercised in the past week (0–7), and 2) whether gone outside to walk, hike, and exercise in the past week (0 = no, 1 = yes), capturing both the frequency and the context (e.g., outdoor) of physical activity. Mental health is indicated using the four-item Patient Health Questionnaire (PHQ-4). We used multilevel Within-Between Random Effects models (REWB) that provide coefficients for both between-individual as well as within-individual associations. ResultsWe reported three main findings. First, both within-person and between-person increases in physical activity are significantly associated with lower levels of anxiety and depression regardless of the measure. Second, between-person changes in physical activity are more strongly associated with mental health than within-person changes. Third, a large share of the between-person association (about 32%) can be attributed to structural inequalities captured by status markers such as gender, race, and SES. ConclusionsOur findings indicate the value of physical activity as a mental health intervention and highlight that both between-individual inequality in physical activity, as well as within-individual changes in physical activity, should be considered. The finding of a stronger between-person association that is largely related to status markers such as gender, race, and SES can be used to develop targeted intervention and prevention strategies to improve mental health through physical activity among the most vulnerable populations.

Location and home rules of cannabis use – Findings from marijuana use and environmental survey 2020, a nationally representative survey in the United States

Cannabis combustion and aerosolization may be associated with adverse health for users and nonusers through secondhand and thirdhand exposure. As cannabis regulation becomes more lenient, understanding where cannabis is used and whether homes have rules restricting use is needed. This study aimed to identify locations, presence of other people, and in-home rules of cannabis use in the United States (U.S.). This secondary analysis of 3,464 inhalation-based (smoking, vaping, dabbing) cannabis users in past 12 months drew from a cross-sectional probability-based online panel of 21,903 U.S. adults in early 2020, providing nationally representative estimates. We describe presence of other people and location at most recent use (smoking, vaping, dabbing, respectively). We also describe household restrictions on in-home cannabis smoking by cannabis smokers vs non-smokers, and by presence of children in home. Cannabis smoking, vaping, and dabbing most often occurred at users’ own homes (65.7%, 56.8%, and 46.9%, respectively). More than 60% of smoking, vaping, and dabbing occurred with someone else present. About 68% of inhalation-based cannabis users (70% and 55%, smokers and non-smokers, respectively) did not have complete restrictions on in-home cannabis smoking; among them, over a quarter lived with children under 18. In the U.S., inhalation-based cannabis use most commonly occurs at home, with others present and a substantial proportion of users lacking complete in-home cannabis smoking restrictions, raising risks of secondhand and thirdhand smoke exposure. These circumstances demand residential interventions for developing bans on indoor cannabis smoking, especially around vulnerable children.

Can Survey Scales Affect What People Report as A Fair Income? Evidence From the Cross-National Probability-Based Online Panel CRONOS

Can Survey Scales Affect What People Report as A Fair Income? Evidence From the Cross-National Probability-Based Online Panel CRONOS

Social-psychological Aspects of Probability-based Online Panel Participation

Abstract In this study, we use qualitative research methods to identify, discuss, and investigate the self-reported motivational factors and barriers in all stages of the probability-based online panel lifecycle—recruitment to the panel, wave-by-wave data collection, and voluntary attrition. Our data were gathered with qualitative in-depth interviews (IDIs). Between March 2020 and February 2021, panelists from the Life in Australia™ probability online panel were classified into four groups based on their previous panel response behavior and each was interviewed. The qualitative data were used to link the reported motivation for and barriers against joining and staying active in the panel with theories about survey participation. Evidence from the IDIs shows that motivations and barriers can be directly linked to social-psychological theories that explain panel/longitudinal survey participation, including how theories such as social-exchange theory, leverage-salience theory, and the reasoned action approach, are sufficiently robust to help understand the time dimension of survey participation and behavioral change of panel members. Our findings have practical implications for probability-based online panel management.

How Weighting by Past Vote Can Improve Estimates of Voting Intentions

Polling error for the 2020 US election was the highest in 40 years and no mode of surveying was unambiguously more accurate. This occurred amid several recent polling failures in other countries. Online panels, as the dominant method now used by pollsters to survey voters, are well-positioned to help reduce the level of bias in pre-election polls. Here, we present a case for those pollsters using online panels for pre-election polling to (re)consider using past vote choice (i.e., whom respondents voted for in the previous election) as a weighting variable capable of reducing bias in their election forecasts under the right circumstances. Our data are from an Australian pre-election poll, conducted on a probability-based online panel one month prior to the 2019 Australian federal election. Three different measures of recalled vote choice for the 2016 election were used in weighting the forecast of the 2019 election outcome. These were (1) a short-term measure of recall for the 2016 vote choice obtained three months after the 2016 election, (2) a long-term measure obtained from the same panelists three years after the 2016 election and (3) a hybrid measure with a random half of panelists allocated their short-term past vote measure for 2016 and the remainder their long-term measure. We then examined the impacts on the bias and variance of the resulting estimates of the 2019 voting intentions. Using the short-term measure of the 2016 recalled vote choice in our weighting significantly reduced the bias of the resulting 2019 voting intentions forecast, with an acceptable impact on variance, and produced less biased estimates than when using either of the other two past vote measures. The short-term recall measure also generally resulted in better estimates than a weighting approach that did not include any past vote adjustment. Implications for panel providers are discussed.

Equipping the Offline Population with Internet Access in an Online Panel: Does It Make a Difference?

Abstract Online panel surveys are often criticized for their inability to cover the offline population, potentially resulting in coverage error. Previous research has demonstrated that non-internet users in fact differ from online individuals on several sociodemographic characteristics. In attempts to reduce coverage error due to missing the offline population, several probability-based online panels equip offline households with an internet connection and a simple computer or tablet. However, the question remains whether the recruitment of offline individuals for an online panel leads to substantial changes in survey estimates. That is, it is unclear whether estimates derived from the survey data are affected by the differences between the groups of online and offline individuals. Against this background, we investigate how the inclusion of the previously offline population into the German Internet Panel affects various survey estimates such as voting behavior and social engagement. Overall, we find little evidence for the claim that equipping otherwise offline individuals with online access affects the estimates derived from previously online individuals only.

Evaluation of a protocol for eliciting narrative accounts of pediatric inpatient experiences of care.

To evaluate the measurement properties of a set of six items designed to elicit narrative accounts of pediatric inpatient experience. Data came from 163 participants recruited from a probability-based online panel of U.S. adults. Participants were family members of a child who had an overnight hospital stay in the past 12 months. Cross-sectional survey with follow-up phone interviews. Participants completed an online (n=129) or phone (n=34) survey about their child's hospitalization experience. The survey contained closed-ended items from the Child Hospital Consumer Assessment of Healthcare Providers and Systems (Child HCAHPS) survey, followed by the six narrative items. Approximately 2 weeks after completing the survey, 47 participants additionally completed a one-hour, semi-structured phone interview, the results of which served as a "gold standard" for evaluating the fidelity of narrative responses. Qualitative content analysis was used to code narrative and interview responses for domains of patient experience and actionability. The average narrative was 248 words (SD=319). Seventy-nine percent of narratives mentioned a topic included in the Child HCAHPS survey; 89% mentioned a topic not covered by that survey; and 75% included at least one detailed description of an actionable event. Overall, there was 66% correspondence between narrative and interview responses. Correspondence was higher on the phone than in the online condition (75% vs. 59%). Narratives elicited from rigorously designed multi-item sets can provide detailed, substantive information about pediatric inpatient experiences that hospitals could use to improve child and family experiences during pediatric hospitalization. They add context to closed-ended survey item responses and provide information about experiences of care important to children and families that are not included in quantitative surveys.

Relationship Between Past Survey Burden and Response Probability to a New Survey in a Probability-Based Online Panel

Abstract We conducted an idiographic analysis to examine the effect of survey burden, measured by the length of the most recent questionnaire, or number of survey invitations (survey frequency) in a one-year period preceding a new survey, on the response probability to a new survey in a probability-based Internet panel. The individual response process was modeled by a latent Markov chain with questionnaire length and survey frequency as explanatory variables. The individual estimates were obtained using a Monte Carlo based method and then pooled to derive estimates of the overall relationships and to identify specific subgroups whose responses were more likely to be impacted by questionnaire length or survey frequency. The results show an overall positive relationship between questionnaire length and response probability, and no significant relationship between survey frequency and response probability. Further analysis showed that longer questionnaires were more likely to be associated with decreased response rates among racial/ethnic minorities and introverted participants. Frequent surveys were more likely to be associated with decreased response rates among participants with a large household. We discuss the implications for panel management and advocate targeted interventions for the small subgroups whose response probability may be negatively impacted by longer questionnaires or frequent surveys.

Response Burden and Dropout in a Probability-Based Online Panel Study – A Comparison between an App and Browser-Based Design

Abstract Survey respondents can complete web surveys using different Internet-enabled devices (PCs versus mobile phones and tablets) and using different software (web browser versus a mobile software application, “app”). Previous research has found that completing questionnaires via a browser on mobile devices can lead to higher breakoff rates and reduced measurement quality compared to using PCs, especially where questionnaires have not been adapted for mobile administration. A key explanation is that using a mobile browser is more burdensome and less enjoyable for respondents. There are reasons to assume apps should perform better than browsers, but so far, there have been few attempts to assess this empirically. In this study, we investigate variation in experienced burden across device and software in wave 1 of a three-wave panel study, comparing an app with a browser-based survey, in which sample members were encouraged to use a mobile device. We also assess device/software effects on participation at wave 2. We find that compared to mobile browser respondents, app respondents were less likely to drop out of the study after the first wave and the effect of the device used was mediated by subjective burden experienced during wave 1.

Impact of the COVID-19 Pandemic on Women's Health Care Access: A Cross-Sectional Study.

Background: There has been limited study of how the COVID-19 pandemic has affected women's health care access. Our study aims to examine the prevalence and correlates of COVID-19-related disruptions to (1) primary care; (2) gynecologic care; and (3) preventive health care among women. Materials and Methods: We recruited 4,000 participants from a probability-based online panel. We conducted four multinomial logistic regression models, one for each of the study outcomes: (1) primary care access; (2) gynecologic care access; (3) patient-initiated disruptions to preventive visits; and (4) provider-initiated disruptions to preventive visits. Results: The sample included 1,285 women. One in four women (28.5%) reported that the pandemic affected their primary care access. Sexual minority women (SMW) (odds ratios [OR]: 1.67; 95% confidence intervals [CI]: 1.19-2.33) had higher odds of reporting pandemic-related effects on primary care access compared to women identifying as heterosexual. Cancer survivors (OR: 2.07; 95% CI: 1.25-3.42) had higher odds of reporting pandemic-related effects on primary care access compared to women without a cancer history. About 16% of women reported that the pandemic affected their gynecologic care access. Women with a cancer history (OR: 2.34; 95% CI: 1.35-4.08) had higher odds of reporting pandemic-related effects on gynecologic care compared to women without a cancer history. SMW were more likely to report patient- and provider-initiated delays in preventive health care. Other factors that affected health care access included income, insurance status, and having a usual source of care. Conclusions: The COVID-19 pandemic disrupted women's health care access and disproportionately affected access among SMW and women with a cancer history, suggesting that targeted interventions may be needed to ensure adequate health care access during the COVID-19 pandemic.

What about the Less IT Literate? A Comparison of Different Postal Recruitment Strategies to an Online Panel of the General Population

Even though the proportion of individuals who are not equipped to participate in online surveys is constantly decreasing, many surveys face an under-representation of individuals who do not feel IT literate enough to participate. Using experimental data from a probability-based online panel, we study which recruitment survey mode strategy performs best in recruiting less IT-literate persons for an online panel. The sampled individuals received postal invitations to conduct the recruitment survey in a self-completion mode. We experimentally vary four recruitment survey mode strategies: one online mode strategy, two sequential mixed-mode strategies, and one concurrent mode strategy. We find the recruitment survey mode strategies to have a major effect on the sample composition of the recruitment survey, but the differences between the strategies vanish once respondents are asked to proceed with the panel online.

Modeling Group-Specific Interviewer Effects on Survey Participation Using Separate Coding for Random Slopes in Multilevel Models

Abstract Despite its importance in terms of survey participation, the literature is sparse on how face-to-face interviewers differentially affect specific groups of sample units. This paper demonstrates how an alternative parametrization of the random components in multilevel models, so-called separate coding, delivers valuable insights into differential interviewer effects for specific groups of sample members. In the example of a face-to-face recruitment interview for a probability-based online panel, we detect small interviewer effects regarding survey participation for non-Internet households, whereas we find sizable interviewer effects for Internet households. We derive practical guidance for survey practitioners to address differential interviewer effects based on the proposed variance decomposition.

Data Privacy Concerns as a Source of Resistance to Complete Mobile Data Collection Tasks Via a Smartphone App

Abstract Smartphones present many interesting opportunities for survey research, particularly through the use of mobile data collection applications (apps). There is still much to learn, however, about how to integrate apps in general population surveys. Recent studies investigating hypothetical willingness to complete mobile data collection tasks via an app suggest there may be substantial resistance, in particular, due to concerns around data privacy. There is not much evidence about how privacy concerns influence actual decisions to participate in app-based surveys. Theoretical approaches to understanding privacy concerns and survey participation decisions would suggest that the influence of the former over the latter is likely to vary situationally. In this paper, we present results from a methodological experiment conducted in the context of a three-wave probability-based online panel survey of the general population as part of the 2019 Swiss Election Study (“Selects”) testing different ways of recruiting participants to an app. Questions included at wave 1 about online data privacy concerns and comfort sharing different types of data with academic researchers allow us to assess their impact on both hypothetical willingness to download a survey app for completing questionnaires, to take and share photos, and to share the smartphone’s GPS location and actual completion of these tasks. Our findings confirm that general concerns about online data privacy do influence hypothetical willingness to complete mobile data collection tasks, but may be overridden by how comfortable people feel about sharing specific types of data with researchers. When it comes to actual compliance with task requests, however, neither privacy concerns nor comfort sharing data seem to matter. We conclude with recommendations for exploring these relationships further in future app-based studies.

The Early Bird Catches the Worm! Setting a Deadline for Online Panel Recruitment Incentives

The literature on the effects of incentives in survey research is vast and covers a diversity of survey modes. The mode of probability-based online panels, however, is still young and so is research into how to best recruit sample units into the panel. This paper sheds light on the effectiveness of a specific type of incentive in this context: a monetary incentive that is paid conditionally upon panel registration within two weeks of receiving the initial postal mail invitation. We tested early bird cash incentives in a large-scale recruitment experiment for the German Internet Panel (GIP) in 2018. We find that panel response rates are significantly higher when offering early bird cash incentives and that fieldwork progresses considerably faster, leading to fewer reminders and greater cost-effectiveness. Furthermore, sample representativeness is similarly high with or without early bird incentives.

The power of online panel paradata to predict unit nonresponse and voluntary attrition in a longitudinal design

The objective of this study is to identify factors affecting participation rates, i.e., nonresponse and voluntary attrition rates, and their predictive power in a probability-based online panel. Participation for this panel had already been investigated in the literature according to the socio-demographic and socio-psychological characteristics of respondents and different types of paradata, such as device type or questionnaire navigation, had also been explored. In this study, the predictive power of online panel participation paradata was instead evaluated, which was expected (at least in theory) to offer even more complex insight into respondents’ behavior over time. This kind of paradata would also enable the derivation of longitudinal variables measuring respondents’ panel activity, such as survey outcome rates and consecutive waves with a particular survey outcome prior to a wave (e.g., response, noncontact, refusal), and could also be used in models controlling for unobserved heterogeneity. Using the Life in Australia™ participation data for all recruited members for the first 30 waves, multiple linear, binary logistic and panel random-effect logit regression analyses were carried out to assess socio-demographic and online panel paradata predictors of nonresponse and attrition that were available and contributed to the accuracy of prediction and the best statistical modeling. The proposed approach with the derived paradata predictors and random-effect logistic regression proved to be reasonably accurate for predicting nonresponse—with just 15 waves of online panel paradata (even without sociodemographics) and logit random-effect modeling almost four out of five nonrespondents could be correctly identified in the subsequent wave.

Enhancing Participation in Probability-Based Online Panels: Two Incentive Experiments and Their Effects on Response and Panel Recruitment

This article investigates how mail-based online panel recruitment can be facilitated through incentives. The analysis relies on two incentive experiments and their effects on panel recruitment, and the intermediate participation in the recruitment survey. The experiments were implemented in the context of the German Emigration and Remigration Panel Study and encompass two samples of randomly sampled persons. Tested incentives include a conditional lottery, conditional monetary incentives, and the combination of unconditional money-in-hand with conditional monetary incentives. For an encompassing evaluation of the link between incentives and panel recruitment, the article further assesses the incentives’ implications for demographic composition and panel recruitment unit costs. Multivariate analysis indicates that low combined incentives (€5/€5) or, where unconditional disbursement is unfeasible, high conditional incentives (€20) are most effective in enhancing panel participation. In terms of demographic bias, low combined incentives (€5/€5) and €10 conditional incentives are the favored options. The budget options from the perspective of panel recruitment include the lottery and the €10 conditional incentive which break-even at net sample sizes of 1000.