Crosswise Model Research Articles

Reducing Turnout Misreporting in Online Surveys

Assessing individual-level theories of electoral participation requires survey-based measures of turnout. Yet, due to a combination of sampling problems and respondent misreporting, postelection surveys routinely overestimate turnout, often by large margins. Using an online survey experiment fielded after the 2015 British general election, we implement three alternative survey questions aimed at correcting for turnout misreporting and test them against a standard direct turnout question used in postelection studies. Comparing estimated to actual turnout rates, we find that while all question designs overestimate aggregate turnout, the item-count technique alleviates the misreporting problem substantially, whereas a direct turnout question with additional face-saving options and a crosswise model design help little or not at all. Also, regression models of turnout estimated using the item-count measure yield substantively similar inferences regarding the correlates of electoral participation to models estimated using “gold-standard” validated vote measures. These findings stand in contrast to those suggesting that item-count techniques do not help with misreporting in an online setting and are particularly relevant given the increasing use of online surveys in election studies.

Detecting nonadherence without loss in efficiency: A simple extension of the crosswise model.

In surveys concerning sensitive behavior or attitudes, respondents often do not answer truthfully, because of social desirability bias. To elicit more honest responding, the randomized-response (RR) technique aims at increasing perceived and actual anonymity by prompting respondents to answer with a randomly modified and thus uninformative response. In the crosswise model, as a particularly promising variant of the RR, this is achieved by adding a second, nonsensitive question and by prompting respondents to answer both questions jointly. Despite increased privacy protection and empirically higher prevalence estimates of socially undesirable behaviors, evidence also suggests that some respondents might still not adhere to the instructions, in turn leading to questionable results. Herein we propose an extension of the crosswise model (ECWM) that makes it possible to detect several types of response biases with adequate power in realistic sample sizes. Importantly, the ECWM allows for testing the validity of the model's assumptions without any loss in statistical efficiency. Finally, we provide an empirical example supporting the usefulness of the ECWM.

Uncovering a Blind Spot in Sensitive Question Research: False Positives Undermine the Crosswise-Model RRT

Validly measuring sensitive issues such as norm violations or stigmatizing traits through self-reports in surveys is often problematic. Special techniques for sensitive questions like the Randomized Response Technique (RRT) and, among its variants, the recent crosswise model should generate more honest answers by providing full response privacy. Different types of validation studies have examined whether these techniques actually improve data validity, with varying results. Yet, most of these studies did not consider the possibility of false positives, i.e., that respondents are misclassified as having a sensitive trait even though they actually do not. Assuming that respondents only falsely deny but never falsely admit possessing a sensitive trait, higher prevalence estimates have typically been interpreted as more valid estimates. If false positives occur, however, conclusions drawn under this assumption might be misleading. We present a comparative validation design that is able to detect false positives without the need for an individual-level validation criterion — which is often unavailable. Results show that the most widely used crosswise-model implementation produced false positives to a nonignorable extent. This defect was not revealed by several previous validation studies that did not consider false positives — apparently a blind spot in past sensitive question research.

Sensitive questions in online surveys : an experimental evaluation of different implementations of the randomized response technique and the crosswise model

Self-administered online surveys may provide a higher level of privacy protection to respondents than surveys administered by an interviewer. Yet, studies indicate that asking sensitive questions is problematic also in self-administered surveys. Because respondents might not be willing to reveal the truth and provide answers that are subject to social desirability bias, the validity of prevalence estimates of sensitive behaviors from online surveys can be challenged. A well-known method to overcome these problems is the Randomized Response Technique (RRT). However, convincing evidence that the RRT provides more valid estimates than direct questioning in online surveys is still lacking. We therefore conducted an experimental study in which different implementations of the RRT, including two implementations of the so-called crosswise model, were tested and compared to direct questioning. Our study is an online survey (N = 6,037) on sensitive behaviors by students such as cheating in exams and plagiarism. Results vary considerably between different implementations, indicating that practical details have a strong effect on the performance of the RRT. Among all tested implementations, including direct questioning, the unrelated-question crosswise-model RRT yielded the highest estimates of student misconduct.

On the comprehensibility and perceived privacy protection of indirect questioning techniques.

On surveys that assess sensitive personal attributes, indirect questioning aims at increasing respondents' willingness to answer truthfully by protecting confidentiality. However, the assumption that subjects understand questioning procedures fully and trust them to protect their privacy is rarely tested. In a scenario-based design, we compared four indirect questioning procedures in terms of their comprehensibility and perceived privacy protection. All indirect questioning techniques were found to be less comprehensible by respondents than a conventional direct question used for comparison. Less-educated respondents experienced more difficulties when confronted with any indirect questioning technique. Regardless of education, the crosswise model was found to be the most comprehensible among the four indirect methods. Indirect questioning in general was perceived to increase privacy protection in comparison to a direct question. Unexpectedly, comprehension and perceived privacy protection did not correlate. We recommend assessing these factors separately in future evaluations of indirect questioning.

The Frequency of High-Risk Behaviors Among Iranian College Students Using Indirect Methods: Network Scale-Up and Crosswise Model.

BackgroundSince the direct questions usually underestimate the frequency of sensitive behaviors, indirect methods can be used to estimate the frequency of some risky behaviors such as illicit drug use, sexual behaviors especially where these behaviors are highly stigmatized.ObjectivesIn the current study, we indirectly estimated the prevalence of some risky behaviors among college students using two indirect methods: network scale-up (NSU) and crosswise model (CM).Patients and MethodsHaving recruited 563 students from one of Iran’s major medical universities, the prevalence of opium and drug use, alcohol consumption, relationships with the opposite sex (RWOS),and extra/pre-marital sex (EPMS) were estimated using two indirect methods.ResultsThe estimated prevalence using the CM and NSU were alcohol consumption (16.8% vs. 8.1%), opium use (2.2% both), methamphetamine use (7.2% vs. 1.2%), taking tramadol without medical indications (14.8% vs. 4.8%), RWOS (42.3% vs. 31.9%), and EPMS (12.4% vs. 7.1%).ConclusionsLower estimations in the NSU method might be due to the transmission barrier, which means that students were not fully aware of the high-risk behaviors of their close friends. Nonetheless, it seems that these risky behaviors were more or less common among Iranian college students.

Re-assessing the merits of measuring tax evasion through business surveys: an application of the crosswise model

Self-assessments by respondents in surveys are often the only available measure of tax evasion in developing countries at the microeconomic level. However, they suffer from the reluctance of respondents to reveal their own illicit behavior. This paper evaluates whether this weakness of self-assessments can at least partially be overcome through a novel questioning method, the crosswise model, which allows estimating the prevalence of tax evasion, but not identifying whether the individual respondent engages in tax evasion or not. Using evidence from Serbia, we show that crosswise model-based estimates of the share of firms which significantly underreport sales exceed those obtained from conventional methods by around 10 % points or more. With respect to wage underreporting to evade payroll tax and social security contributions, we do not find differences. These results appear to be robust to a number of modifications, and we explore various potential causes that lead to these results.

Knowledge, attitude, self-efficacy and estimation of frequency of condom use among Iranian students based on a crosswise model.

This study aimed to evaluate the knowledge, attitude and self-efficacy and condom use among Iranian students. Knowledge, attitudes towards condoms and self-efficacy of condom use questionnaire were translated and back translated. Then content validity ratio (CVR) and content validity index (CVI) questions higher than 0.6 and 0.71 were accepted, respectively. A "crosswise model" was used to estimate condom use. After pretest on 10 students, reliability test was done by test-re-test and Cronbach's α. A multi stage sampling was done for 1500 students in Shahroud universities. They filled in a self administered questionnaire and gave it back to the researcher after 40-50 min. Knowledge, in females was (3.03±1.1) and in males was (3.53±0.83) (p=0.001). Also a small percentage of the students, i.e. 83 girls (9.1%) and six boys (1.1%) had good self-efficacy of condom use (p=0.001). Estimates relating to the use of condoms with a crosswise test showed that 16% of girls (CI=16, 24) and 27% of boys (CI=17, 38) had used a condom during intercourse. It seems that these three important indicators of reproductive health should be incorporated in AIDS control programs and training programs should emphasize it more.

When to Protect? Using the Crosswise Model to Integrate Protected and Direct Responses in Surveys of Sensitive Behavior

Sensitive survey techniques (SSTs) are frequently used to study sensitive behaviors. However, existing strategies for employing SSTs lead to highly variable prevalence estimates and do not permit analysts to address the question of whether the use of an SST is actually necessary. The current article presents a survey questioning strategy and corresponding statistical framework that fills this gap. By jointly analyzing survey responses generated by an SST (the crosswise model) along with direct responses about the sensitive behavior, the article's framework addresses the question of whether the use of an SST is required to study a given sensitive behavior, provides an efficient estimate of the prevalence of the sensitive behavior, and, in its extended form, efficiently estimates how individual characteristics relate to the likelihood of engaging in the behavior. The utility of the approach is demonstrated through an examination of gender differences in proclivities towards corruption in Costa Rica.

A Strong Validation of the Crosswise Model Using Experimentally-Induced Cheating Behavior.

We constructed an online cheating paradigm that could be used to validate the Crosswise Model ( Yu, Tian, & Tang, 2008 ), a promising indirect questioning technique designed to control for socially desirable responding on sensitive questions. Participants qualified for a reward only if they could identify the target words from three anagrams, one of which was virtually unsolvable as shown on a pretest. Of the 664 participants, 15.5% overreported their performance and were categorized as cheaters. When participants were asked to report whether they had cheated, a conventional direct question resulted in a substantial underestimate (5.1%) of the known prevalence of cheaters. Using a CWM question resulted in a more accurate estimate (13.0%). This result shows that the CWM can be used to control for socially desirable responding and provides estimates that are much closer to the known prevalence of a sensitive personal attribute than those obtained using a direct question.

Assessing the validity of two indirect questioning techniques: A Stochastic Lie Detector versus the Crosswise Model.

Estimates of the prevalence of sensitive attributes obtained through direct questions are prone to being distorted by untruthful responding. Indirect questioning procedures such as the Randomized Response Technique (RRT) aim to control for the influence of social desirability bias. However, even on RRT surveys, some participants may disobey the instructions in an attempt to conceal their true status. In the present study, we experimentally compared the validity of two competing indirect questioning techniques that presumably offer a solution to the problem of nonadherent respondents: the Stochastic Lie Detector and the Crosswise Model. For two sensitive attributes, both techniques met the "more is better" criterion. Their application resulted in higher, and thus presumably more valid, prevalence estimates than a direct question. Only the Crosswise Model, however, adequately estimated the known prevalence of a nonsensitive control attribute.

Crosswise Model to Assess Sensitive Issues: A Study on Prevalence of Drug Abuse Among University Students of Iran.

Background:Precise assessment of the prevalence of illicit drug use, face various methodological challenges.Objectives:The current study aimed to investigate the prevalence of illicit drug use among students studying at Universities in Shahroud (Northeast of Iran) through crosswise method.Patients and Methods:Participants of this cross-sectional study were 1646 students at Universities in Shahroud. The data collection instrument was a questionnaire designed based on crosswise model (CM).Results:The results of the study showed that 19% of students used at least one kind of illicit drug and 14.9% used opium residue once in their life. Moreover, 3.5% of students used drug and 3% used opium residue during the last month; 40.3% of the students admitted that they fully understood the instruction of the questionnaire; 9.6% said that did not comprehend the instruction at all, and 38.1% believed they partly understood the instruction. The result showed that 33.7% of the students fully trusted, 39.8% partly trusted, and 18.4% poorly trusted this method. There was a significant relationship between comprehension level and trust in CM (P < 0.001).Conclusions:This method can be appropriate to estimate sensitive issues; however, lack of understanding the method or doubting the confidentiality of the responses can lead to bias in the results. That is to say, if the instructions are introduced better, the level of trusting the method will increase and accordingly more reliable responses can be obtained. More studies under controlled conditions are required to interpret the findings of the current study better.

Estimating the prevalence of researcher misconduct: a study of UK academics within biological sciences.

Misconduct in academic research is undoubtedly increasing, but studies estimating the prevalence of such behaviour suffer from biases inherent in researching sensitive topics. We compared the unmatched-count technique (UCT) and the crosswise-model (CM), two methods specifically designed to increase honest reporting to sensitive questions, with direct questioning (DQ) for five types of misconduct in the biological sciences. UCT performed better than CM and either outperformed or produced similar estimates to DQ depending on the question. Estimates of academic misconduct increased with decreasing seriousness of the behaviour, from c. 0% for data fabrication to >68% for inappropriate co-authorship. Results show that research into even minor issues of misconduct, is sensitive, suggesting that future studies should consider using specialised questioning techniques as they are more likely to yield accurate figures.

Applying 'Benford's Law' to the Crosswise Model: Findings from an Online Survey on Tax Evasion

Many surveys on sensitive topics such as tax evasion suffer from the reluctance of respondents to provide truthful answers which can cause downward-biased estimates. This paper addresses this problem by making use of a recent survey method (Crosswise Model) designed to provide positive incentives for respondents to answer sensitive questions more truthful. We extend the Crosswise Model by applying the so-called “Benford Illusion” which allows us to increase the precision of the Crosswise Model that is less statistically efficient than other methods. To test the effectiveness of the model in providing privacy protection, we carried out an online survey in which the respondents were randomly allocated into two splits differing only by the questioning technique applied. Our results suggest that the Crosswise Model can help to increase privacy protection compared to a simple direct questioning approach. As a consequence, survey estimates of tax evasion using the Crosswise Model are likely to become more valid. At the same time, we show that were able to obtain an efficient estimator without substantially decreasing privacy protection, even for a relatively small sample size.

Measuring and explaining tax evasion: Improving self-reports using the crosswise model

Due to its sensitive nature, tax compliance is difficult to study empirically, and valid information on tax evasion is rare. More specifically, when directly asked on surveys, respondents are likely to underreport their evasion behavior. Such invalid responses not only bias prevalence estimates but may also obscure associations with individual predictors. To generate more valid estimates of tax evasion, we used a new method of data collection for sensitive questions, the crosswise model (CM). The CM is conceptually based on the randomized response technique (RRT), but due to its advanced design, it is better suited for large surveys than classical RRTs. In an experimental online survey, we compared the CM (N=862) to standard direct questioning (DQ; N=305). First, our results showed that the CM was able to elicit a higher proportion of self-stigmatizing reports of tax evasion by increasing privacy in the data collection process. Second, on average, we found stronger effects of our predictor variables on tax evasion in the CM condition compared with the DQ condition such that an egoistic personality and the opportunity for tax evasion predicted actual tax evasion only in the CM condition.

Estimating the Prevalence of Illicit Drug Use Among Students Using the Crosswise Model

Objective: The aim of our study is to compare the prevalence of illicit drug use estimated through a technique referred to as the “crosswise model” (CM) with the results from conventional direct questioning (DQ). Method: About 1,500 students from Tehran University of Medical Sciences 2009–2010 were first interviewed by DQ and, then three months later, by the CM. Result: The CM yielded significantly higher estimates than DQ for lifetime prevalence of use of any illicit drug (CM = 20.2%,DQ = 3.0%, p < .001) and for lifetime prevalence of use of opium or its residue (CM = 13.6%, DQ = 1.0%, p < .001). Also, for use of any illicit drug in the last month and use of opium or its residue in the last month, the CM yielded higher point estimates than DQ, although these differences were not significant (any drug: CM = 1.5%, DQ = 0.2%, p = .66; opium: CM = 3.8%, DQ = 0.0%, p = .21). Conclusion: Our findings suggest that the CM is a fruitful data collection method for sensitive topics such as substance abuse.

Estimation of sexual behavior in the 18-to-24-years-old Iranian youth based on a crosswise model study

BackgroundIn many countries, negative social attitude towards sensitive issues such as sexual behavior has resulted in false and invalid data concerning this issue.This is an analytical cross-sectional study, in which a total number of 1500 single students from universities of Shahroud City were sampled using a multi stage technique. The students were assured that their information disclosed for the researcher will be treated as private and confidential. The results were analyzed using crosswise model, Crosswise Regression, T-test and Chi-square tests.FindingsIt seems that the prevalence of sexual behavior among Iranian youth is 41% (CI = 36-53).ConclusionFindings showed that estimation sexual relationship in Iranian single youth is high. Thus, devising training models according to the Islamic-Iranian culture is necessary in order to prevent risky sexual behavior.

Re-Assessing the Merits of Measuring Tax Evasions through Surveys: Evidence from Serbian Firms

This paper addresses the major weakness of measuring tax evasion through business and household surveys, namely the reluctance of respondents to answer truthfully due to the threat of disclosure. First, we assess the merits of a novel questioning method to gather information about tax evasion by means of business surveys. This approach allows estimating the prevalence of tax evasion, but it does not allow identifying whether the individual firm engages in tax evasion or not, therefore providing incentives for survey participants to answer truthfully. Second and contrary to most other business surveys, we differentiate between two common modes of tax evasion, namely underreporting of sales and informal supplements to official wages (‘envelope wages’). Using evidence from Serbia, we show that the estimated share of firms which underreport sales and wages, respectively, by at least 10% is higher under the crosswise model compared to the case when conventional questioning methods applied in business surveys such as the World Bank Enterprise Surveys are used. However, the difference is only significant with respect to sales. These results appear to be robust to a number of modifications, and we explore various potential causes that lead to these results.

Asking sensitive questions: A statistical power analysis of randomized response models.

This article derives the power curves for a Wald test that can be applied to randomized response models when small prevalence rates must be assessed (e.g., detecting doping behavior among elite athletes). These curves enable the assessment of the statistical power that is associated with each model (e.g., Warner's model, crosswise model, unrelated question model, forced-choice models, item count model, cheater detection model). This power analysis can help in choosing the optimal model and sample size and in setting model parameters in survey studies. The general framework can be applied to all existing randomized response model versions. The Appendix of this article contains worked-out numerical examples to demonstrate the power analysis for each specific model.

Asking Sensitive Questions Using the Crosswise Model: An Experimental Survey Measuring Plagiarism

Yu, Tian, and Tang (2008) proposed two new techniques for asking questions on sensitive topics in population surveys: the triangular model (TM) and the crosswise model (CM). The two models can be used as alternatives to the well-known randomized response technique (RRT) and are meant to overcome some of the drawbacks of the RRT. Although Yu, Tian, and Tang provide a promising theoretical analysis of the proposed models, they did not test them. We therefore provide results from an experimental survey in which the crosswise model was implemented and compared to direct questioning. To our knowledge, this is the first empirical evaluation of the crosswise model. We focused on the crosswise model because it seems better suited than the triangular model to overcome the self-protective “no” bias observed for the RRT. This paper-and-pencil survey on plagiarism was administered to Swiss and German students in university classrooms. Results suggest that the CM is a promising data-collection instrument eliciting more socially undesirable answers than direct questioning.