Assessment Of Non-response Bias Research Articles

National Pharmacist Workforce Study (NPWS): Description of 2019 Survey Methods and Assessment of Nonresponse Bias.

National Pharmacist Workforce Studies (NPWS) have been conducted in the U.S. every five years since 2000. This article describes the online survey methods used for the latest NPWS conducted in 2019 and provides an assessment for nonresponse bias. Three waves of emails containing a link to the online survey were sent to a random sample of about 96,000 pharmacists licensed in the United States. The survey asked about pharmacist employment, work activities, work–life balance, practice characteristics, pharmacist demographics and training. A total of 5467 usable responses were received, for a usable response rate of 5.8%. To assess for nonresponse bias, respondent characteristics were compared to the population of U.S. pharmacists and a benchmark, while a wave analysis compared early and late respondents. The pharmacist sample–population comparison and the benchmark comparison showed that the NPWS respondents had a higher percentage of female pharmacists and a lower proportion of young pharmacists compared to the population of U.S. pharmacists and the benchmark sample. In some contrast, the wave analysis showed that the early respondents had a higher percentage of males and older pharmacists compared to the late respondents. Both the wave analysis and the benchmark comparison showed that the NPWS respondents (and early respondents) had a lower percent of pharmacists with a PharmD degree than did the late respondents and the benchmark group. These differences should be considered when interpreting the findings from the 2019 NPWS.

Demographic and socioeconomic characteristics of Canadian medical students: a cross-sectional study

BackgroundWhile the importance of medical students’ demographic characteristics in influencing the scope and location of their future practice is recognized, these data are not systematically collected in Canada. This study aimed to characterize and compare the demographics of Canadian medical students with the Canadian population.MethodsThrough an online survey, delivered in 2018, medical students at 14 English-speaking Canadian medical schools provided their age, sex, gender identity, ethnicity, educational background, and rurality of the area they grew up in. Respondents also provided information on parental income, occupation, and education as markers of socioeconomic status. Data were compared to the 2016 Canadian Census.ResultsA total of 1388 students responded to the survey, representing a response rate of 16.6%. Most respondents identified as women (63.1%) and were born after 1989 (82.1%). Respondents were less likely, compared to the Canadian Census population, to identify as black (1.7% vs 6.4%) (P < 0.001) or Aboriginal (3.5% vs. 7.4%) (P < 0.001), and have grown up in a rural area (6.4% vs. 18.7%) (P < 0.001). Respondents had higher socioeconomic status, indicated by parental education (29.0% of respondents’ parents had a master’s or doctoral degree, compared to 6.6% of Canadians aged 45–64), occupation (59.7% of respondents’ parents were high-level managers or professionals, compared to 19.2% of Canadians aged 45–64), and income (62.9% of respondents grew up in households with income >$100,000/year, compared to 32.4% of Canadians). Assessment of non-response bias showed that our sample was representative of all students at English-speaking Canadian medical schools with respect to age, though a higher proportion of respondents were female. Additionally, there were no differences between early and late respondents with respect to ethnicity, rurality, and parental income, occupation, and education.ConclusionsCanadian medical students have different socioeconomic characteristics compared to the Canadian population. Collecting and analyzing these characteristics can inform evidence-based admissions policies.

Technical Note: Recommendations for Assessing Unit Nonresponse Bias in Dyadic Focused Empirical Supply Chain Management Research

ABSTRACTThe last decade has seen an increase in empirical supply chain management research with dyadic data. Such data structures can further complicate the assessment of nonresponse bias, which plays a key role in establishing the credibility of research results. A survey of 75 research articles with dyadic data, published in five empirically focused supply chain management academic journals, over the last decade, reveals a lack of agreement on methods used in the assessment for potential unit nonresponse bias. Of the various statistical tests found, only the Multivariate Analysis of Variance (MANOVA) approach allows for a single statistical test to be utilized in assessing for potential unit nonresponse bias via incorporation of the design structure of the dyadic data. We investigate the use of an effect size confidence interval coverage, of a MANOVA, to detect a meaningful difference between respondents and nonrespondents correctly. Our results show that with dyadic data, such meaningful differences can be detected with significantly smaller sample size requirements than traditional approaches such as t‐tests or ANOVA. Recommendations are provided for setting up and executing a MANOVA to assess for potential unit nonresponse bias with dyadic data.

Assessment of Non-Response Bias in Estimates of Alcohol Consumption: Applying the Continuum of Resistance Model in a General Population Survey in England.

BackgroundPrevious studies have shown heavier drinkers are less likely to respond to surveys and require extended efforts to recruit. This study applies the continuum of resistance model to explore how survey estimates of alcohol consumption may be affected by non-response bias in three consecutive years of a general population survey in England.MethodsUsing the Health Survey for England (HSE) survey years 2011–13, number of contact attempts (1–6 and 7+) were explored by socio-demographic and drinking characteristics. The odds of drinking more than various thresholds were modelled using logistic regression. Assuming that non-participants were similar to those who were difficult to contact (the continuum of resistance model), the effect of non-response on measures of drinking was investigated.ResultsIn the fully-adjusted regression model, women who required 7+ calls were significantly more likely to drink more than the UK Government’s recommended daily limit (OR 1.19, 95% CI 1.06–1.33, P = 0.003) and to engage in heavy episodic drinking (OR 1.23, 95% CI 1.07–1.42, P = 0.004), however this was not significant in men in the fully-adjusted model. When the continuum of resistance model was applied, there was an increase in average weekly alcohol consumption of 1.8 units among men (a 12.6% relative increase), and an increase of 1.5 units among women (a 20.5% relative increase). There was also an increase in the prevalence of heavy episodic drinking of 2.5% among men (an 12.0% relative increase) and of 2.0% among women (a 15.8% relative increase), although other measures of drinking were less affected.ConclusionOverall alcohol consumption and the prevalence of heavy episodic drinking were higher among HSE participants who required more extended efforts to contact. The continuum of resistance model suggests non-response bias does affect survey estimates of alcohol consumption.

Non-response bias assessment in logistics survey research: use fewer tests?

Purpose – The purpose of this paper is to consider the concepts of individual and complete statistical power used for multiple testing and shows their relevance for determining the number of statistical tests to perform when assessing non-response bias. Design/methodology/approach – A statistical power analysis of 55 survey-based research papers published in three prestigious logistics journals (International Journal of Physical Distribution and Logistics Management, Journal of Business Logistics, Transportation Journal) over the last decade was conducted. Findings – Results show that some of the low complete power levels encountered could have been avoided if fewer tests had been used in the assessment of non-response bias. Originality/value – The research offers important recommendations to scholars engaged in survey research as they assess the effects of non-respondents on research findings. By following the recommended strategies for testing non-response bias, researchers can improve the statistical power of their findings.

Guidelines for Improving the Power Values of Statistical Tests for Nonresponse Bias Assessment in OM Research

ABSTRACTThe assessment of nonresponse bias in survey‐based empirical studies plays an important role in establishing the credibility of research results. Statistical methods that involve the comparison of responses from two groups (e.g., early vs. late respondents) on multiple characteristics, which are relevant to the study, are frequently utilized in the assessment of nonresponse bias. We consider the concepts of individual and complete statistical power used for multiple testing and show their relevance for determining the number of statistical tests to perform when assessing nonresponse bias. Our analysis of factors that influence both individual and complete power levels, yielded recommendations that can be used by operations management (OM) empirical researchers to improve their assessment of nonresponse bias. A power analysis of 61 survey‐based research papers published in three prestigious academic operations management journals, over the last decade, showed the occurrence of very low (&lt;0.4) power levels in some of the statistical tests used for assessing nonresponse bias. Such low power levels can lead to erroneous conclusions about nonresponse bias, and are indicators of the need for more rigor in the assessment of nonresponse bias in OM research.

Estimating the Effect of Nonresponse Bias in a Survey of Hospital Organizations

Nonresponse bias in survey research can result in misleading or inaccurate findings and assessment of nonresponse bias is advocated to determine response sample representativeness. Four methods of assessing nonresponse bias (analysis of known characteristics of a population, subsampling of nonresponders, wave analysis, and linear extrapolation) were applied to the results of a postal survey of U.K. hospital organizations. The purpose was to establish whether validated methods for assessing nonresponse bias at the individual level can be successfully applied to an organizational level survey. The aim of the initial survey was to investigate trends in the implementation of radiographer abnormality detection schemes, and a response rate of 63.7% (325/510) was achieved. This study identified conflicting trends in the outcomes of analysis of nonresponse bias between the different methods applied and we were unable to validate the continuum of resistance theory as applied to organizational survey data. Further work is required to ensure established nonresponse bias analysis approaches can be successfully applied to organizational survey data. Until then, it is suggested that a combination of methods should be used to enhance the rigor of survey analysis.

International health policy survey in 11 countries: assessment of non-response bias in the Norwegian sample

BackgroundInternational health policy surveys are used to compare and evaluate health system performance, but little is known about the effects of non-response. The objective of this study was to assess the effects of non-response in the Norwegian part of the Commonwealth Fund international health policy survey in 2009.MethodsAs part of an international health policy survey in 2009 a cross-sectional survey was conducted in Norway among a representative sample of Norwegian general practitioners. 1 400 randomly selected GPs were sent a postal questionnaire including questions about the Norwegian health care system, the quality of the GPs' own practice and the cooperation with specialist health care. The survey included three postal reminders and a telephone follow-up of postal non-respondents. The main outcome measures were increase in response rate for each reminder, the effects of demographic and practice variables on response, the effects of non-response on survey estimates, and the cost-effectiveness of each reminder.ResultsAfter three postal reminders and one telephone follow-up, the response rate was 59.1%. Statistically significant differences between respondents and non-respondents were found for three variables; group vs. solo practice (p = 0.01), being a specialist or not (p < 0.001) and municipality centrality (least central vs. most central, p = 0.03). However, demographic and practice variables had little association with five outcome variables and the overall survey estimates changed little with additional reminders. In addition, the cost-effectiveness of the final reminders was poor.ConclusionsThe response rate in the Norwegian survey was satisfactory, and the effect of non-response was small indicating adequate representativeness. The cost-effectiveness of the final reminders was poor. The Norwegian findings strengthen the international project, but restrictions in generalizability warrant further study in other countries.

Response rate and nonresponse bias in a questionnaire survey of dentists

(a) To report on response rate and nonresponse bias of a questionnaire survey of dentists. (b) To make recommendations for future questionnaire survey research in dentistry. A questionnaire was mailed to a stratified systematic sample of 908 Australian dentists. The strategy included three mailings, a final telephonic contact, university stationery, paid reply envelopes and personalized correspondence. Nonresponse bias was assessed by comparing responses to a simple 'yes/no' question from each contact (late responders), and by comparing demographic information (nonresponders). The response rate achieved was 87% and there was no evidence of nonresponse bias based on practice location or year of graduation. The cumulative proportions of 'yes/no' responses essentially remained constant after each contact, but significantly more late responders answered in the negative to the test question than did early responders. The telephonic contact aided in the identification of nonparticipants and ineligible units. The current survey indicates that differences in data between early and nonresponders can occur despite there being no demographic differences. Therefore, assessment of nonresponse bias based on demographic data alone would seem to be insufficient. Questionnaire survey research must first be based on sound sampling techniques, and then on achieving as high a response rate as possible using the many incentives available.

PEDAKSI: Methodology for collecting data about survey non-respondents

The effects of unit non-response on survey errors are of great concern to researchers.However, direct assessment of non-response bias in survey estimates is rarely possible.Attempts are often made to adjust for the effects of non-response by weighting, but thisusually relies on the use of frame data or external population data, which are at bestmodestly correlated with the survey variables. This paper reports the development ofa method to collect limited survey data from non-respondents to personal interviewsurveys and a large-scale field test of the method on the British Crime Survey (BCS).The method is shown to be acceptable and low cost, to provide valid data, and to haveno detrimental effect on the main survey. The use of the resultant data to estimatenon-response bias is illustrated and some substantive conclusions are drawn for the BCS.

Assessment of non-response bias in a probability household survey of male same-gender sexual behavior.

To assess non-participation bias in a survey of male sexual behavior. A household survey was carried out in 1992-1993 using a probability sampling frame in Mexico City. Demographic variables were available for all eligible men. The extent of non-participation bias was estimated using a version of the Heckman method, which utilizes two equations, one to predict participation and the other to predict reports of same-gender sexual behavior. A total of 8,068 of the 13,713 eligible men completed a face-to-face questionnaire (response rate 59%); 173 men (2.1%) reported bisexual behavior in their lifetime, and 37 (0.4%) reported only male partners. Survey participation was predicted using demographic variables: 67% of the observations were correctly predicted by a probit regression model: 82% of participants and 53% of non-participants (pseudo-r2 = 0.13). Same-gender sexual behavior was predicted by variables indicating attachment to gay/bisexual social networks, history of sexually transmitted diseases, positive attitudes towards gay and bisexual males, and lack of support from male relatives. Ninety-seven per cent of the cases was correctly predicted by the probit model (pseudo-r2 = 0.14). The correlation between these two equations was not statistically significant. These results indicate that prevalence estimates of same-gender sexual behavior among Mexico City men were not biased by selective survey participation. Careful selection and training of household interviewers may have assisted in minimizing potential bias.

Validation of a Population Screening Questionnaire to Assess Prevalence of Stroke

A cross-sectional survey is the best method for determining the prevalence of a chronic condition such as stroke. Before embarking on a large population study, a valid screening instrument is necessary. This study aimed to validate a self-completion postal screening questionnaire for assessing lifetime history of stroke. A random sample of 2000 people aged 45 years and over was selected from the Family Health Services Authority register. A brief self-completion questionnaire requesting self-reported history of stroke was mailed to each individual. Responders reporting a history of stroke received a request for a standardized home assessment visit. Confirmation of the diagnosis of stroke was made based on information from the home visit together with multiple sources of case ascertainment. False-negative responses and assessment of nonresponse bias were documented. The response rate for the questionnaire was 88%. Of the sample, 120 (6%) who were wrongly registered were excluded; 173 (10%) of the 1663 responders reported a history of stroke. The question "Have you ever had a stroke?" had a sensitivity of 95% and a specificity of 96%. This simple self-completion questionnaire is a valid means of screening for cases of stroke in the community and could form the basis for subsequent studies of prevalence and health needs.