Item Nonresponse Research Articles

Multivariate Small-Area Estimation for Mixed-type Response Variables with Item Nonresponse

Abstract Many surveys collect information on discrete characteristics and continuous variables, that is, mixed-type variables. Small-area statistics of interest include means or proportions of the response variables as well as their domain means, which are the mean values at each level of a different categorical variable. However, item nonresponse in survey data increases the complexity of small-area estimation. To address this issue, we propose a multivariate mixed-effects model for mixed-type response variables subject to item nonresponse. We apply this method to two data structures where the data are missing completely at random by design. We use empirical data from two separate studies: a survey of pet owners and a dataset from the National Resources Inventory. In these applications, our proposed method leads to improvements relative to a direct estimator and a predictor based on a univariate model.

Effects of Changing Modes on Item Nonresponse in Panel Surveys

Abstract To investigate the effect of a change from the telephone to the web mode on item nonresponse in panel surveys, we use experimental data from a two-wave panel survey. The treatment group changed from the telephone to the web mode after the first wave, while the control group continued in the telephone mode. We find that when changing to the web, “don’t know” answers increase moderately from a low level, while item refusal increases substantially from a very low level. This is the case for all person groups, although socio-demographic characteristics have some additional effects on giving a don’t know or a refusal when changing mode.

범주형 전환문항을 포함하는 항목 무응답 대체 방법 비교

In social surveys, unfolding bracket questions are used as a technique to reduce the loss of information due to nonresponse when item nonresponses occur. Item nonresponse, including unfolding bracket questions, has a similar structure to interval censored survival data. However, imputation methods to handle item nonresponses including unfolding bracket questions are limited. In this study, we examine performance of imputation with and without unfolding bracket questions. Also we compare three imputation methods to handle unfolding bracket questions. The First method is a linear regression-based nearest neighbor hotdeck imputation. Secondly, it is possible to consider a uniform distribution that selects a value with the same probability within the section for each individual. The third is an imputation method using the survival function estimated by the nonparametric maximum likelihood estimation method. Hotdeck imputation was considered as an imputation for not using unfolding bracket questions. The performance of these imputation methods was evaluated through simulation and the KLoSA study is used to provide examples of the application of these methods to real data. Imputation methods using unfolding bracket questions showed better performance than the method without unfolding bracket questions. The performance of the imputation method using the survival function estimated by the nonparametric maximum likelihood estimation method was better when the nonresponse rate was low and when the nonresponse rate is large in terms of root mean square error. When the non-response rate is large, the linear regression-based nearest neighbor hotdeck imputation method showed better results in terms of mean bias.

Impacts of cultural factors and mode of administration on item nonresponse for political questions in the European context

ABSTRACT We tested the impacts of Hofstede’s cultural factors and mode of administration on item nonresponse (INR) for political questions in the European Values Study (EVS). We worked with the integrated European Values Study dataset, using descriptive analysis and multilevel binary logistic regression models. We concluded that (1) modes of administration with an interviewer lead to an INR lower than those without an interviewer. In terms of Hofstede’s cultural factors, we concluded that (2) the higher the power distance index in the country, the lower the INR, (3) the higher the rate of individualism in the country, the higher the INR, and (4) the higher the uncertainty avoidance index, the lower the INR. These findings stress the importance of sensitive work with cross-national data, especially with increasingly abundant data from mixed-mode surveys.

Item non-response imputation in the Korea National Health and Nutrition Examination Survey.

The Korea National Health and Nutrition Examination Survey (KNHANES) is a public health survey that assesses individuals' health and nutritional status and monitors the prevalence of major chronic diseases. In general, sampling weights are adjusted for unit non-responses and imputation is conducted for item non-responses. In this study, we proposed strategies for imputing item non-responses in the KNHANES in order to improve the usefulness of data, minimize bias, and increase statistical power. After applying logical imputation, we adopted 2 separate imputation methods for each variable type: unweighted sequential hot-deck imputation for categorical variables and sequential regression imputation for continuous variables. For variance estimation, multiple imputations were applied to the continuous variables. To evaluate the performance of the proposed strategies, we compared the marginal distributions of imputed variables and the results of multivariable regression analysis for the complete-case data and the expanded data with imputed values, respectively. When comparing the marginal distributions, most non-responses were imputed. The multivariable regression coefficients presented similar estimates; however, the standard errors decreased, resulting in statistically significant p-values. The proposed imputation strategies may cope with the loss of precision due to missing data, thus enhancing statistical power in analyses of the KNHANES by providing expanded data with imputed values. The proposed imputation strategy may enhance the utility of data by increasing the number of complete cases and reducing the bias in the analysis, thus laying a foundation to cope with the occurrence of item non-responses in further surveys.

Effects of Question Characteristics on Item Nonresponse in Telephone and Web Survey Modes

Telephone surveys face more and more criticism because of decreasing coverage and increasing costs, and the risk of producing socially desirable answers. Consequently, survey administrators consider switching their surveys to the web mode, although the web mode is more susceptible to item nonresponse. Still, we do not know whether this is true for all question types. In this article, we analyze to what extent item nonresponse depends on question characteristics such as their form or difficulty in the telephone and the web mode. We use data from an experiment in which individuals randomly sampled from a population register are experimentally assigned to these two modes. Distinguishing effects on the frequency of don’t know responses, item refusals, and mid-scale responding, we find more don’t know responses and item refusals for the web mode generally, but no differences for mid-scale responding. However, this relationship depends on the characteristics of the question.

Assessing and Comparing Data Imputation Techniques for Item Nonresponse in Household Travel Surveys

This research provides a comparative assessment of data imputation techniques for item nonresponse in household travel surveys. Using the Transportation Tomorrow Survey (TTS) data for the Region of Waterloo in Ontario, Canada, a series of synthetic datasets are generated with varying amounts of missing data, while preserving the respective proportions of missing items and missing item combinations in the original survey data. Then, the performances of six different imputation techniques are compared. The six different imputation techniques include two simple imputation techniques (mode and hot-deck), three discriminative models (logistic regression, multi-layered perceptron, support vector machines) and one generative model (autoencoder). This assessment compares these techniques, as well as the impact of the proportion of item nonresponse in the dataset through their repeated application to multiple synthetic datasets. Results show that the machine/deep learning techniques (both generative and discriminative) not previously applied to household travel survey data outperform their simple imputation counterparts. Overall, the accuracy of travel household survey data imputation is shown to depend on many factors, including the technique employed, the dimensionality of the missing item, and the hypertuning of the technique (if applicable), but not on the amount of missing data in these experiments. This research should prove beneficial to practitioners who often confront item nonresponse in their household travel survey data by providing evidence and recommendations to support the selection and implementation of a data imputation technique. The research methodology also provides a repeatable procedure for future researchers to test data imputation techniques on their own datasets.

Analysis of Determinants on Item Nonresponse:A Study on Household/individual Income and Sexual Intercourse Nonresponse

This study examines the factors influencing item nonresponse, especially when it comes to income and sexual behavior, in the Japanese context. Many item nonresponses on questions on income and sexual behavior are a serious problem for analysis since they might cause biased estimations. However, many previous studies on item nonresponse when it comes to income and sexual issues examine surveys carried out in Western societies. Moreover, many previous studies have not distinguished between “Refused” and “Don't Know.” In addition, many previous studies on income nonresponse do not focus on the effect of religion, although questions on religion have been argued to affect survey cooperation. Thus, we analyzed the data from the Japanese General Social Survey 2000/2001 to distinguish the reason for nonresponse properly, reveal factors on income and sexual behavior nonresponses, and examined the effect of religion. We conducted multinomial logistic regressions of nonresponse in household income, individual income, and sexual behavior. Our findings are as follows: (1) Factors that affected nonresponses of income and sexual behavior are different between Japanese and Western societies. (2) The effect of age and educational background are different between “Refused” and “Don't Know.” (3) Respondents whose family has a religion tended to answer with household income. These findings indicate that we should consider the nonresponse mechanism due to the Japanese-specific context, distinguish the reason for nonresponse because of different effects of factors, and focus on the effect of religion on income nonresponse.

Associations Between Privacy-Related Constructs and Depression and Suicide Risk in Health Care Professionals, Trainees, and Students.

The University of California, San Diego screens health care professionals, trainees, and students for depression and suicide risk. Individuals complete a voluntary, anonymous online screening tool and choose whether to provide personal demographic information. This study assessed the relationship between privacy-related constructs and self-rated depression and suicide risk. The authors analyzed responses to the screening tool collected from January 2018 to December 2019. Measures of depression, suicidal ideation and behaviors, and worry about stigma for seeking mental health services (i.e., privacy-related concern) were gathered. The number of demographic item nonresponses (i.e., age, gender, ethnicity/race, professional position) was operationalized as privacy-related behavior. Linear and logistic regression models were used to determine associations between privacy-related constructs (concern and behavior) and depression and suicide measures. A total of 1,224 respondents were included. On average, respondents reported mild depression (mean = 9.12, standard deviation = 5.94), but 43% (524/1,224) reported at least moderate depression. One in 5 respondents (248/1,224) reported worry about stigma for seeking mental health services, and more than 17% (212/1,224) skipped at least 1 demographic question. Privacy-related concern was statistically significantly and positively associated with recent depression and suicidal ideation and behaviors (odds ratios [ORs] = 3.13-7.02; 95% confidence interval [CI], 2.23-19.20; P's < .001) and with lifetime suicide attempts (OR = 1.76; 95% CI, 1.08-2.86; P = .02). Privacy-related behavior was statistically significantly and positively associated with suicide action (OR = 2.23; 95% CI, 1.24-4.02; P = .008). Privacy-related constructs may be meaningful correlates of mental health as respondents who endorsed these constructs had increased odds of worse depression and suicidal ideation and behaviors. Considering privacy-related constructs may be useful for identifying health care professionals, trainees, and students experiencing distress and in need of imminent mental health resources.

A New Electorate? Explaining the Party Preferences of Immigrant-Origin Voters at the 2017 Bundestag Election

AbstractImmigrants now constitute a sizeable and rapidly growing group among many Western countries' electorates, but analyses of their party preferences remain limited. Theoretically, immigrants' party preferences might be explained with both standard electoral theories and immigrant-specific approaches. In this article, we rigorously test both perspectives against each other using the most recent data from Germany. Applying the Michigan model, with its three central explanatory variables – party identification, issue orientations and candidate evaluations – to the party preferences of immigrant-origin and native voters, we find that this standard model can explain both groups well. In contrast, we find no direct effects of the most prominent immigrant-specific variables, and neither do these meaningfully moderate the Michigan variables. However, we find strong formative effects on the presence of political attitudes and beliefs: immigrants with a longer time spent in Germany, a stronger German identity and less experience of discrimination report significantly fewer item non-responses for the Michigan model's main explanatory variables.

More Clarification, Less Item Nonresponse in Establishment Surveys? A Split-Ballot Experiment

The IAB Job Vacancy Survey of the German Institute for Employment Research collects detailed information on job search and vacancy durations for an establishment’s last successful hiring process. The duration questions themselves are burdensome for respondents to answer as they ask for precise dates of the earliest possible hiring for the vacancy, the start of the personnel search, and the decision to hire the selected applicant. Consequently, the nonresponse rates for these items have been relatively high over the years (up to 21 percent). In an effort to reduce item nonresponse, a split-ballot experiment was conducted to test the strategy of providing additional clarifying information and examples to assist respondents in answering the date questions. The results revealed a backfiring effect. Although there was evidence that respondents read the additional clarifying information, this led to even more item nonresponse and lower data quality compared to the control group. Additionally, we observed a negative spillover effect with regard to item nonresponse on a subsequent (non-treated) question. We conclude this article by discussing possible causes of these results and suggestions for further research.

A Mixture Response Time Process Model for Aberrant Behaviors and Item Nonresponses

In high-stakes, large-scale, standardized tests with certain time limits, examinees are likely to engage in either one of the three types of behavior (e.g., van der Linden & Guo, 2008; Wang & Xu, 2015): solution behavior, rapid guessing behavior, and cheating behavior. Oftentimes examinees do not always solve all items due to various reasons such as time limit or test-taking strategy. Item nonresponses may happen due to intentionally omitting some items (omitted responses) or due to lack of time (not-reached responses). Both types are related to latent abilities and hence the missingness is nonignorable. In this article, we proposed an innovative mixture response time process model (1) to detect two most common aberrant behaviors: rapid guessing behavior and cheating behavior, and (2) to account for two types of item nonresponses: not-reached items and omitted items. The new model combines the two-stage approach of Wang et al. (2018) with Lu and Wang (2020) model. It also contains two steps: (1) a mixture response time process model is first fitted to the responses and response times data to distinguish normal and aberrant behaviors and to account for the missing data mechanism; and (2) a Bayesian residual index is used to further distinguish rapid guessing and cheating behaviors. Simulation results show that the two-stage method yields accurate item and person parameter estimates, as well as high detection of aberrant behaviors. A real data analysis was conducted to illustrate the potential application of the proposed method.

The Effects of Question, Respondent and Interviewer Characteristics on Two Types of Item Nonresponse

Abstract In this article, we examine two types of item nonresponse in a face-to-face population survey: ‘don’t know’ (DK) and ‘item refusal’ (REF). Based on the cognitive model of survey response, the theory of survey satisficing and previous research, we derive explanatory variables on three levels: question, respondent and interviewer characteristics. The results of our cross-classified model show that while the two levels question and respondents’ characteristics affected both types of item nonresponse, interviewer characteristics affected only DK answers. Our results also confirm that DK and REF are substantially different item nonresponse types resulting from distinguishable disruptions of the cognitive response process. Since most results are in line with prior theoretical predictions, they suggest that survey practitioners are well-advised by continuing to follow the large body of practical guidance derived from the theories tested here.

P216 Assessing patient satisfaction with the cystic fibrosis telehealth service

Objectives: Telehealth (via videocall) was rapidly implemented by the South Australian Adult Cystic Fibrosis (CF) Service in March 2020 due to the Novel Coronavirus 2019 pandemic. Home spirometers were distributed as part of objective patient assessment. Our aim was to assess patient satisfaction with the telehealth service in CF multidisciplinary clinic via online survey. Methods: All patients of the CF service were invited to complete an online survey designed to assess satisfaction with telehealth, which was undertaken through the consumer organisation Cystic Fibrosis South Australia. A quantitatively driven concurrent mixed-methods design was chosen, with statistical analysis of symmetrical five-point Likert responses using IBM SPSS Statistics for Windows v.26 and coding of unstructured responses for thematic analysis. Results: To date 57/160 patients (35.6%) completed the survey, with some item non-responses. 37/57 (64.9%) participants were aged ≤35 years of age, 42/53 (79.2%) had ≥2 telehealth appointments, and 46/53 (86.7%) found it easy/very easy to access. 18/48 (37.5%) of participants had technical issues. Despite this 48/52 (92.3%) felt satisfied/very satisfied with engagement of CF team via telehealth. 38/50 (76%) found telehealth more convenient with significant time saved. The thematic analysis added depth to quantitative analysis by identification of six key themes: convenience, ease of access, maintaining the therapeutic relationship, comprehensive care, safety, and patient centred care. Conclusion: Most patients reported high satisfaction with telehealth and would like a hybrid clinic with the option of telehealth or in person review for clinics into the future.

An Empirical Investigation of Factors Contributing to Item Nonresponse in Self-Reported Bullying Instruments

ABSTRACT Researchers often use self-reported instruments to collect data from students when investigating the causes and effects of bullying. When completing the instrument, students may skip items despite anonymous data collection. To interpret data accurately, researchers must identify the causes of nonresponse. This study examined the impact of item (types of bullying and position) and student (sex and grade level) factors on nonresponse in bullying items. The sample included 10,681 students from grades 8, 9, and 11 who skipped at least one bullying item in the Minnesota Student Survey. Results showed that students were more likely to skip the items presented toward the end of the survey. The type of bullying had no impact on nonresponse. Male students and students at higher grade levels were more likely to skip the items. To alleviate the nonresponse issue, researchers are recommended to consider both survey length and student factors when administering bullying instruments.

A Response Time Process Model for Not‐Reached and Omitted Items

AbstractItem nonresponses are prevalent in standardized testing. They happen either when students fail to reach the end of a test due to a time limit or quitting, or when students choose to omit some items strategically. Oftentimes, item nonresponses are nonrandom, and hence, the missing data mechanism needs to be properly modeled. In this paper, we proposed to use an innovative item response time model as a cohesive missing data model to account for the two most common item nonresponses: not‐reached items and omitted items. In particular, the new model builds on a behavior process interpretation: a person chooses to skip an item if the required effort exceeds the implicit time the person allocates to the item (Lee &amp; Ying, 2015; Wolf, Smith, &amp; Birnbaum, 1995), whereas a person fails to reach the end of the test due to lack of time. This assumption was verified by analyzing the 2015 PISA computer‐based mathematics data. Simulation studies were conducted to further evaluate the performance of the proposed Bayesian estimation algorithm for the new model and to compare the new model with a recently proposed “speed‐accuracy + omission” model (Ulitzsch, von Davier, &amp; Pohl, 2019). Results revealed that all model parameters could recover properly, and inadequately accounting for missing data caused biased item and person parameter estimates.

Telephone versus web panel National Survey for monitoring adoption of preventive behaviors to climate change in populations: a case study of Lyme disease in Qu\xe9bec, Canada

BackgroundTo monitor the adoption of climate change adaptive behaviors in the population, public health authorities have to conduct national surveys, which can help them target vulnerable subpopulations. To ensure reliable estimates of the adoption of these preventive behaviors, many data collection methods are offered by polling firms. The aim of this study was to compare a telephone survey with a web survey on Lyme disease with regard to their representativeness.MethodsThe data comes from a cross-sectional study conducted in the Province of Québec (Canada). In total, 1003 people completed the questionnaire by telephone and 956 filled in a web questionnaire. We compared the data obtained from both survey modes with the census data in regard to various demographic characteristics. We then compared the data from both samples in terms of self-reported Lyme disease preventive behaviors and other theoretically associated constructs. We also assessed the measurement invariance (equivalence) of the index of Lyme disease preventive behaviors across the telephone and web samples.ResultsFindings showed that neither the telephone nor the web panel modes of data collection can be considered more representative of the target population. The results showed that the proportion of item non-responses was significantly higher with the web questionnaire (5.6%) than with the telephone survey (1.3%), and that the magnitude of the differences between the two survey modes was nil for 19 out of the 30 items related to Lyme disease, and small for 11 of them. Results from invariance analyses confirmed the measurement invariance of an index of adaptation to Lyme disease, as well as the mean invariance across both samples.ConclusionsOur results suggested that both samples provided similar estimates of the level of adaptation to Lyme disease preventive behaviors. In sum, the results of our study showed that neither survey mode was superior to the other. Thus, in studies where adaptation to climate change is monitored over time, using a web survey instead of a telephone survey could be more cost-effective, and researchers should consider doing so in future surveys on adaptation to climate. However, we recommend conducting a pretest study before deciding whether to use both survey modes or only one of them.

The association between interviewer gender and responses to sensitive survey questions in a sample of Haitian women

ABSTRACTBias may be introduced in survey data collection when participants answer questions differently depending on interviewer gender. This could affect the validity of collected data, especially sensitive data. Using sexual behavior data collected in a 2017–2018 cross-sectional survey of Haitian women (n = 304), we evaluated the associations between interviewer-gender and three different outcomes: (1) question-specific response rates, (2) total number of non-responses, and (3) differences in reported answers. We observed higher item response rates for sensitive sexual behavior questions when the interviewer was female and more item non-responses (Don’t know/Refuse) when the interviewer was male. Among those who did respond, participants were more likely to report some sensitive sexual behaviors to women and others to men. We conclude that researchers should consider the sociocultural norms of the study population and the potential for interviewer gender bias in the planning and analysis phase of studies using self-reported data.

Effects of age and cognitive function on data quality of standardized surveys in nursing home populations

BackgroundData quality is of special concern when it comes to survey research in nursing homes. Very little is known about specifics of cognitively impaired elderly in responding to survey questions. This study examines effects of cognitive impairment, age, gender, and interview duration on the data quality in a sample of 659 nursing home residents (NHR).MethodsWithin a cross-sectional design, survey methodology was used to evaluate the pain situation in 13 nursing homes. Residents were stratified into NHR with no/mild (Mini-Mental State Examination MMSE: 18–30) and NHR with moderate (MMSE: 10–17) cognitive impairment. Data quality is measured by item nonresponse (INR). Correlation analyses, ANCOVA, linear and logistic regression models are applied.ResultsNeither interview duration nor gender have effects on item nonresponse. Age accounts for higher INR (β = 0.12, p < 0.001). Cognitive impairment strongly predicts INR (β = − 0.40, p < 0.001). INR significantly differs between NHR with no/mild (3.98%) and moderate cognitive impairment (11.85%). The likelihood of INR > 5% for residents with moderate cognitive impairment is 3.8-times (p < 0.001) of that for those with no/mild impairment.ConclusionsSurveys are adequate for residents with no/mild cognitive impairment but data quality is threatened in residents with moderate impairments. Precision and validity of responses from NHR with progressed cognitive impairment are potentially limited and results may be biased. The results clearly do support the need for a multidisciplinary ‘general theory’ of the question−/answer-process which has to be also inclusive for cognitively impaired elderly persons.

What They Expect Is What You Get: The Role of Interviewer Expectations in Nonresponse to Income and Asset Questions.

Personal income and assets are sensitive topics to discuss. This phenomenon is reflected in high rates of nonresponse to financial questions in surveys. In face-to-face surveys, item nonresponse is influenced by interviewers. Although interviewers are trained to conduct standardized interviews, some obtain a higher number of item nonresponses than others. This study examines interviewer effects on nonresponse to questions about household income, bank balances, and interest and dividend income in the Survey of Health, Ageing and Retirement in Europe (SHARE). It first investigates the extent to which interviewers affect nonresponse to income and asset questions and second whether interviewers' prior expectations regarding respondents' likelihood to provide information about their income predict actual nonresponse rates. Results of multilevel modeling show that interviewer influence on nonresponse to the income and asset questions was significant at the five percent level. In addition, interviewer expectations were significantly correlated with "don't know" responses and "refusals." These results indicate that interviewer expectations matter in the context of income and asset questions and that survey practitioners should take this into account when designing interviewer training.