Sensitive Survey Questions Research Articles

Obtaining Answers to Sensitive Survey Questions Using Venn Diagrams

Summary We discuss the application of techniques used to solve Venn diagram survey problems to the problem of collecting truthful responses to sensitive questions. We also develop a method for collecting data for a sensitive question that protects the confidentiality of the respondent’s answer. The method uses techniques that students are taught to solve Venn diagram survey problems, and we discuss a classroom exercise based on the solution.

The fundamental determinants of protest participation: Evidence from Hong Kong’s antiauthoritarian movement

Which fundamental traits are associated with individuals’ participation in antiauthoritarian protests? We conduct a series of surveys eliciting participation in Hong Kong’s antiauthoritarian movement, covering a period that included protests ranging from tens of thousands to over one million participants. For a sample of university students, we construct a comprehensive profile of fundamental economic preferences: risk and time preferences plausibly affecting an individual’s costs of protest participation; social preferences affecting the benefits. We also elicit other fundamental traits: personality, cognitive abilities, and socioeconomic background. We document several facts about protest participants: (i) fundamental economic preferences, particularly risk tolerance and pro-social preferences, are the strongest predictors of protest participation; (ii) the strongest predictors are the same for modest and massive protests, with larger effects for massive protests; (iii) participation in massive protests is not driven by marginal types, but rather by inframarginal types; (iv) both the distribution of fundamental preferences and their relationship with protest participation are very similar between university students and the broader population; and, (v) willingness to respond honestly to sensitive survey questions is high and stable over the entire sample period. Our findings suggest that economic preferences be considered alongside class background and personality as deeply determined traits driving protest participation and can inform the development of dynamic models of protest movements.

A Bayesian Generalized Regression Approach for estimating parameters in the Item Sum Technique of sensitive surveys

Most respondents find it difficult to respond to sensitive questions in surveys especially providing information that require one to disclose behaviors that are possibly against social norms. This study involves solving the problem of non-response or falsification of information in a sample survey as a result of sensitive questions using the Item Sum Technique (IST). The IST is a recent indirect method for collecting data for continuous sensitive characteristics and it is a quantitative variant of the Item Count Technique (ICT). The calibration method has previously been used to estimate the IST, but the method is design-based and only useful for sufficiently large domains. There are situations where the domain sample size is not adequate to meet the needs of the precision of the estimates, hence the smallarea estimation which is model-based may be needed. The Bayesian method is a model-based estimation method that tackles the problem of small area sample size. Results from simulation indicate the efficiency of the Bayesian Generalized Regression (BGREG) method for both small and large domain samples as compared with the Generalized Regression (GREG) and calibration method using three different criteria, AIC, BIC and MSE. A small sample survey on the use of substance abuse and internet gambling among university students from Nigeria was further used to show that the BGREG outperformed the GREG and Calibration methods using the AIC, BIC and MSE criterion.

How Interviewers Affect Responses to Sensitive Questions on the Justification for Wife Beating, the Refusal to have Conjugal Sex, and Domestic Violence in India.

Despite a general understanding that interviewers might cause measurement errors on sensitive questions in sample surveys, there is relatively little research on interviewer effects on responses to questions on women justifying a woman's refusal to have sex with her husband, women justifying wife beating, women's experience of physical and sexual violence, and whether the woman's father ever beat her mother. This study examines interviewer effects on these indicators that were collected in two large-scale National Family Health Surveys (NFHS) in India (2005-2006 and 2015-2016). We use cross-classified random intercept multivariable multilevel logit models to examine interviewer effects. In both surveys, we find large interviewer effects on questions about the justification of a woman refusing to have sex with her husband (32-33% in NFHS-3 and 45-46% in NFHS-4) and the justification of wife beating (27-28% in NFHS-3 and 33-34% in NFHS-4). The interviewer effects were much larger in the 2015-2016 survey than in the 2005-2006 survey. Such large interviewer effects should be considered when interpreting trends and patterns on these topics, especially since the interviewer effects might have changed between survey rounds. Understanding interviewer effects is important given the wide use of these surveys in policy formulation and monitoring in India.

Item Count Technique with a Continuous or Count Control Variable for Analyzing Sensitive Questions in Surveys

Abstract The paper introduces a new item count technique (ICT) for dealing with sensitive features. One of the aims of the proposed method is to eliminate both the “ceiling” and “floor” effects that can occur in the original ICT. In the new model, in contrast to the classical one and recently proposed models, the two groups into which the sample is split are treated similarly. Therefore, there is no traditional distinction between the control and treatment groups. The control variable can be either a qualitative count variable or a continuous variable. To our knowledge, ICT methods with continuous control have not been proposed before. Although our new model differs substantially in the approach, it requires only a slight modification of the questionnaire design, which in no way leaks respondents’ privacy. Substantial gain in statistical efficiency is achieved as compared to the earlier proposed Poisson and negative binomial ICTs. The new method also reduces the bias of maximum likelihood estimators and allows to obtain bootstrap confidence intervals with a substantially smaller width. The statistical properties of the method are investigated theoretically and by simulations.

False Positives and the “More-is-Better” Assumption in Sensitive Question Research

Abstract Several special questioning techniques have been developed in order to counteract misreporting to sensitive survey questions, for example, on criminal behavior. However, doubts have been raised concerning their validity and practical value as well as the strategy of testing their validity using the “more-is-better” assumption in comparative survey experiments. This is because such techniques can be prone to generating false positive estimates, that is, counting “innocent” respondents as “guilty” ones. This article investigates the occurrence of false positive estimates by comparing direct questioning, the crosswise model (CM), and the item count technique (ICT). We analyze data from two online surveys (N = 2,607 and 3,203) carried out in Germany and Switzerland. Respondents answered three questions regarding traits for which it is known that their prevalence in reality is zero. The results show that CM suffers more from false positive estimates than ICT. CM estimates amount to up to 15 percent for a given true value of zero. The mean of the ICT estimates is not significantly different from zero. We further examine factors causing the biased estimates of CM and show that speeding through the questionnaire (random answering) and problems with the measurement procedure—namely regarding the unrelated questions—are responsible. Our findings suggest that CM is problematic and should not be used or evaluated without the possibility of accounting for false positives. For ICT, the issue is less severe.

SENSITIVE QUESTIONS IN SURVEYS

Abstract In research on sensitive questions in surveys, the item count technique (ICT) has gained increased attention in recent years as a means of counteracting the problem of misreporting, that is, the under- and over-reporting of socially undesirable and socially desirable behaviors or attitudes. The performance of ICT compared with conventional direct questioning (DQ) has been investigated in numerous experimental studies, yielding mixed evidence. This calls for a systematic review. For this purpose, the present article reports results from a comprehensive meta-analysis of experimental studies comparing ICT estimates of sensitive items to those obtained via DQ. In total, 89 research articles with 124 distinct samples and 303 effect estimates are analyzed. All studies rely on the “more (less) is better” assumption, meaning that higher (lower) estimates of negatively (positively) connoted traits or behaviors are considered more valid. The results show (1) a significantly positive pooled effect of ICT on the validity of survey responses compared with DQ; (2) a pronounced heterogeneity in study results, indicating uncertainty that ICT would work as intended in future studies; and (3) as meta-regression models indicate, the design and characteristics of studies, items, and ICT procedures affect the success of ICT. There is no evidence for an overestimation of the effect due to publication bias. Our conclusions are that ICT is generally a viable method for measuring sensitive topics in survey studies, but its reliability has to be improved to ensure a more stable performance.

Can you repeat the question? Paradata as a lens to understand respondent experience answering cognitively demanding, sensitive questions.

Survey researchers hope that respondents will provide high-quality data, but evidence suggests that social desirability bias may be commonplace. Social desirability can lead to significant underreporting or overreporting of sensitive behaviors. With better understanding of the cognitive processes that respondents use to prepare and deliver their responses, survey designers could hope to minimize social desirability bias or at least detect settings that lessen its impact. The primary objective of this study was to use survey paradata to understand the psychology of responding to certain types of survey questions. More specifically, we sought to determine how emotional triggering can alter response latencies to cognitively demanding and sensitive survey questions on induced abortion, which is underreported. We hypothesize that having had a prior abortion might lengthen response times to an indirect question about abortion among respondents who have experienced this sensitive reproductive outcome as they hesitate in deciding whether and how to respond to the question. Data come from a representative survey of 6,035 reproductive age women in Rajasthan, India. We used list experiment question active screen time paradata in conjunction with responses from direct questions on abortion to assess our hypothesis. Our final model was a multivariate linear regression with random effects at the level of the interviewer, including adjustments for respondent, community, and interviewer characteristics to estimate within-respondent effects. Results suggest that women who reported an abortion on the direct abortion questions took 5.11 (95% CI 0.21, 10.00) seconds longer to respond to the list experiment treatment list compared to the control list in comparison to women who did not report an abortion on the direct abortion questions. This study demonstrates the additional insights gained when focusing on response latencies to cognitively demanding questions involved in the measurement of sensitive behaviors.

Indirect questioning methods for sensitive survey questions: Modelling criminal behaviours among a prison population.

Information such as the prevalence and frequency of criminal behaviour is difficult to estimate using standard survey techniques because of the tendency of respondents to withhold or misrepresent information. Social desirability bias is a significant threat to the validity of self-reported data, especially when supplied by persons such as sexual offenders or those convicted of theft or substance abuse. The randomized response approach is an alternative to the standard interview method and offers great potential for researchers in the field of criminal justice. By means of a survey of 792 prison inmates, incorporating both indirect and direct response techniques, we investigate if the prison population also has problems recognizing their participation in criminal acts such as theft, illicit drug use, violence against property, reckless driving and arson. Our research findings suggest that self-reported criminal behaviour among a prison population is affected by social desirability bias and that the behaviour considered is significantly associated with the severity of obsessive-compulsive symptoms. The results also demonstrate the inadequacy of traditional, yet widely used, direct questioning methods, and the great potential for indirect questioning techniques to advance policy formation and evaluation in the field of criminal behaviour.

Application of a randomized response technique for the sensitive question survey in female sex workers in Xichang, China.

The technique of sampling is a particularly important aspect of sampling surveys, and the method of determining sample size is the key link in sampling technique. However, a sample size determination method for complex sampling surveys of sensitive issues using a randomized response model is not yet available. In this work, we drew from the theory of probability and mathematical statistics, and developed a series of formulas for sample size determination that are required to estimate the overall proportion and mean on dichotomous sensitive questions in 3-stage sampling using Simmons models. Shortly after generating the formulas, we successfully applied them to a pre-survey regarding sensitive issues among female sex workers (FSWs) in Xichang, and found their practical application to achieve satisfactory effects. By using a large number of Monte Carlo simulation sampling tests, the reliability of the survey methods and formulas for sensitive question surveys were found to be high, but the randomized response technique (RRT) cannot completely eliminate the possibility of social acceptability bias. RRT is an acceptable method for sensitive question surveys within sensitive populations, and this new sampling method is liable to be used for sensitive question surveys of large populations. We believe that the results obtained in the field of public health can provide some data support for clinical practice.

Consequences of Asking Sensitive Questions in Surveys

I review selected articles from the survey methodology literature on the consequences of asking sensitive questions in censuses and surveys, using a total survey error (TSE) framework. I start with definitions of sensitive questions and move to examination of the impact of including sensitive questions on various sources of survey error—specifically, survey respondents’ willingness to participate in a survey (unit nonresponse), their willingness to respond to next rounds of interviews (wave nonresponse), their likelihood to provide an answer to sensitive questions after agreeing to participate in the survey (item nonresponse), and the accuracy of respondents’ answers to sensitive questions (measurement error). I also review the simultaneous impact of sensitive questions on multiple sources of error in survey estimates and discuss strategies to mitigate the impact of asking sensitive questions on measurement errors. I conclude with a summary and suggestions for future research.

Who are the ‘Don’t Knows’? Missing Data in Surveys of Post-Soviet Conflict-affected Regions

Respondents often answer ‘don’t know’ to sensitive survey questions to avoid revealing their true opinions, especially in post-conflict societies, thus requiring difficult decisions for analysing affected survey data. Using the same five sensitive questions in ten surveys from conflict-affected societies in the former Soviet Union in the period 2005–2014, methods for coping with missing data resulting from ‘don’t know’ responses are presented. Many commonly applied missing data treatments are shown to be incompatible with the missing data mechanism for politically sensitive questions, while also significantly affecting statistical results and conclusions. Ultimately, knowledge of local context is paramount for choosing proper missing data treatments.

Eliciting Information from Sensitive Survey Questions

This paper considers how to elicit information from sensitive survey questions. First we thoroughly evaluate list experiments (LE), a leading method in the experimental literature on sensitive questions. Our empirical results demonstrate that the assumptions required to identify sensitive information in LE are violated for the majority of surveys. Next we propose a novel survey method, called Multiple Response Technique (MRT), for eliciting information from sensitive questions. We require all of the respondents to answer three questions related to the sensitive information. This technique recovers sensitive information at a disaggregated level while still allowing arbitrary misreporting in survey responses. An application of the MRT provides novel empirical evidence on sexual orientation and Lesbian, Gay, Bisexual, and Transgender (LGBT)-related sentiment.

The association between interviewer gender and responses to sensitive survey questions in a sample of Haitian women

ABSTRACTBias may be introduced in survey data collection when participants answer questions differently depending on interviewer gender. This could affect the validity of collected data, especially sensitive data. Using sexual behavior data collected in a 2017–2018 cross-sectional survey of Haitian women (n = 304), we evaluated the associations between interviewer-gender and three different outcomes: (1) question-specific response rates, (2) total number of non-responses, and (3) differences in reported answers. We observed higher item response rates for sensitive sexual behavior questions when the interviewer was female and more item non-responses (Don’t know/Refuse) when the interviewer was male. Among those who did respond, participants were more likely to report some sensitive sexual behaviors to women and others to men. We conclude that researchers should consider the sociocultural norms of the study population and the potential for interviewer gender bias in the planning and analysis phase of studies using self-reported data.

Self-reporting discrepancies of bullying victimization and perpetration measures

PurposeTo evaluate the relationship between sociodemographics and the prevalence of bullying victimization and perpetration using single-item and multiple-item measures. MethodsLongitudinal survey data were obtained from 4297 children at fifth, seventh, and tenth grade in three U.S. cities. Bullying victimization and perpetration were measured in two ways: 1) a single-item recall measure; and 2) a separate multiple-item measure using specific behaviors indicating bullying victimization and perpetration. Multilevel logistic regression modeled the relationship between sociodemographics and bullying, stratified by measurement type. ResultsIn fifth grade, 4% of children were identified as victims using the single-item approach but not the multiple-item approach, 27% were identified as victims using the multiple-item approach but not the single-item approach, and 17% were identified as victims using both approaches. For perpetration, 3% were identified using the single-item approach but not the multiple-item approach, 18% were identified using the multiple-item and not the single-item approach, and 4% were identified using both approaches. The odds of victimization were significantly lower in seventh and tenth grades than in fifth grade using both approaches. The single-item odds of perpetration were significantly lower in tenth grade than fifth grade, but the multiple-item odds of perpetration significantly increased over time. ConclusionsBullying prevalence rates are sensitive to the structure of measures. Future research should identify whether these differences reflect a lack of awareness of types of bullying and/or cognitive variability in answering sensitive survey questions.

Sensitive Questions in Cross-National Comparative Surveys

ABSTRACT The article is devoted to the problem of survey items that ask for sensitive information. This factor has a significant impact on the quality and comparability of data from international surveys. We propose a methodology that can be used to comparatively study the level of sensitivity of questions. It is often used in public opinion polls as well as in marketing and academic studies. We were able to use it to identify similarities and differences in what questions are considered sensitive by the different populations of the nine countries of the former Soviet Union as a whole as well as among particular socio-demographic groups. The study analyzes the reasons why certain questions are considered to be sensitive: the violation of various norms, whether legal or political, social or moral, communication etiquette or beliefs about what constitutes private information. We developed a classification of sensitive questions based on the reasons that were identified. The obtained results allow us to better understand the reasons of why certain questions are considered to be sensitive, to develop an approach to how this concept should be defined, and to identify the differences between sensitive questions and those that are perceived to have other qualities, such as those that are “threatening” and questions fraught with social desirability effect. The results of this comparative study can be used to improve the tools and to allow a deeper analysis of cross-national survey data.

The wisdom of crowds design for sensitive survey questions

Survey research on sensitive questions is challenging because respondents often answer untruthfully or completely refuse to answer. Existing indirect questioning techniques address the problem of social desirability bias at the expense of decreasing estimates' efficiency. We suggest the Wisdom of Crowds survey design that does not pose a tradeoff between anonymity and efficiency as an alternative. We outline the conditions necessary for the technique to work and test them empirically. Moreover, we compare the Wisdom of Crowd estimate of a right-wing populist party's vote share to alternative indirect questioning techniques' estimates as well as to the official election result in the 2017 German federal election. Provided its conditions are met, the Wisdom of Crowds design performs best in terms of both bias and efficiency. We conclude that the Wisdom of Crowds design is an important addition to social scientists' survey methodology toolbox.

Sensitive Survey Questions: Measuring Attitudes Regarding Female Genital Cutting Through a List Experiment

AbstractPotential bias in survey responses is higher if sensitive outcomes are measured. This study analyses attitudes towards female genital cutting (FGC) in Ethiopia. A list experiment is designed to elicit truthful answers about FGC support and compares these outcomes with the answers given to a direct question. Our results confirm that the average bias is substantial as answers to direct questions underestimate the FGC support by about 10 percentage points. Moreover, our results provide suggestive but not statistically significant evidence that this bias is more pronounced among uneducated women and women targeted by an NGO intervention (not randomly assigned).

Decomposing the cross-sex misprediction bias of dating behaviors: Do men overestimate or women underreport their sexual intentions?

Men typically predict women's sexual intentions to be higher than women say they are (Haselton & Buss, 2000). It is debated whether this cross-sex bias is because of men overestimating women's intentions (Murray et al., 2017), women underreporting their own intentions (Perilloux & Kurzban, 2015, 2017), or both. To unify the current debate, we decompose the part of the bias attributable to women underreporting versus men overestimating by using a survey method intervention to reduce underreporting of sensitive information: eliciting estimates about others before sensitive self-reports. First, we calibrate the current measurement instrument to assess the overall size of the misprediction bias (Study 1). Then, we manipulate the order-of-elicitation of self- and other-reports (Studies 2 and 3): Women report significantly higher own sexual intentions when they are asked about other targets' intentions before their own, suggesting that 48 to 69% of the overestimation bias is attributable to women underreporting their own sexual intentions. Analogous analyses for the misprediction bias about men suggest that women's overestimation bias of men's sexual intentions is entirely because of men underreporting their own sexual intentions. The findings have important implications for the current debate in the literature on cross-sex misprediction biases and the literature on asking sensitive survey questions. (PsycINFO Database Record

Sensitive Survey Questions with Auxiliary Information

Scholars increasingly rely on indirect questioning techniques to reduce social desirability bias and item nonresponse for sensitive survey questions. The major drawback of these approaches, however, is their inefficiency relative to direct questioning. We show how to improve the statistical analysis of the list experiment, randomized response technique, and endorsement experiment by exploiting auxiliary information on the sensitive trait. We apply the proposed methodology to survey experiments conducted among voters in a controversial antiabortion referendum held during the 2011 Mississippi General Election. By incorporating the official county-level election results, we obtain precinct- and individual-level estimates that are more accurate than standard indirect questioning estimates and occasionally even more efficient than direct questioning. Our simulation studies shed light on the conditions under which our approach can improve the efficiency and robustness of estimates based on indirect questioning techniques. Open-source software is available for implementing the proposed methodology.