Respondent Confusion Research Articles

Evaluasi Kualitas Website KPU Kabupaten Kediri Menggunakan Metode Webqual 4.0 dan Importance Performance Analysis (IPA)

The year 2020 is the year that the dependence with the use of the internet. Due to the pandemic Covid-19 all activities most use the Internet. Includes information on local elections (Elections) simultaneously. As the organizer of the KPU Kediri district also utilize the internet to the success of the dissemination of information and services through a media-based web site or website. Permasa-land in this study originated from the want to know how high people's satisfaction with website KPU Kediri district. People use the website to find information on Local Head Elections (PILKADA). The purpose of this study is to find out whether it is the achievement of services to the satisfaction of the public against the use of the website KPU Kediri district. This research method using Webqual 4.0 and Importance Performance Analysis (IPA). Sample this study uses purposive sampling technique with 100 respondents from the District. Kediri. The results of the research get the value of 93,5% with the value of the gap 0,226 it can be concluded that the level of service quality website KPU Kediri district has not been according to the expectations of society. The results of the Importance Performance Analysis (IPA) shows the mapping of the cartesian diagram is expected to improve the quality of information by providing the information up to date and that the in-formation given on the web site is still not in the appropriate format or still less precise arrangement of the informa-signal. So as to make respondent confusion. Researchers give advice for the management of the website management in preparing a strategy to improve the quality of website KPU Kediri.

An IRT Mixture Model for Rating Scale Confusion Associated with Negatively Worded Items in Measures of Social-Emotional Learning

ABSTRACT We illustrate the application of mixture IRT models to evaluate respondent confusion due to the negative wording of certain items on a social-emotional learning (SEL) assessment. Using actual student self-report ratings on four social-emotional learning scales collected from students in grades 3–12 from CORE Districts in the state of California, we also evaluate the consequences of the potential confusion in biasing student- and school-level scores as well as the estimated correlational relationships between SEL constructs and student-level variables. Models of both full and partial confusion are examined. Our results suggest that (1) rating scale confusion due to negatively worded items does appear to be present; (2) the confusion is most prevalent at lower grade levels (third–fifth); and (3) the occurrence of confusion is positively related to both reading proficiency and ELL status, as anticipated, and consequently biases estimates of SEL correlations with these student-level variables. For these reasons, we suggest future iterations of the SEL measures use only positively oriented items.

Pouring water into wine : revisiting the advantages of the crosswise model for asking sensitive questions

The Crosswise Model (CM) has been proposed as a method to reduce effects of social desirability in sensitive questions. In contrast with former variants of Randomized Response Techniques (RRTs), the crosswise model neither offers a self-protective response strategy, nor does it require a random device. For these reasons, the crosswise model has received a lot of positive attention in the scientific community. However, previous validation studies have mostly analysed negatively connoted behaviour and thus draw on the principle of “more is better”. Higher prevalence rates of socially undesirable behaviour in the crosswise model cannot be attributed unambiguously to a reduction in social desirability bias, since random ticking resulting from respondent confusion about the question format cannot be ruled out as an alternative explanation. Unlike most research on crosswise models and randomized response techniques, we conduct an experiment in a general population survey that does not assess negatively connoted but socially desirable behaviour (namely, whether respondents had donated blood within the last twelve months). This design allows us to empirically disentangle the reduction of social desirability bias from random responses. We find signifcantly higher prevalence rates in the crosswise condition than in the direct question. What is more, we could not identify any subgroup of respondents, in which the CM successfully reduced social desirability bias. These results cast doubts on the validity of cosswise models. They suggest that a considerable number of respondents do not comply with the intended procedure.

Interviewer–respondent Interactions in Conversational and Standardized Interviewing

Standardized interviewing (SI) and conversational interviewing are two approaches to collect survey data that differ in how interviewers address respondent confusion. This article examines interviewer–respondent interactions that occur during these two techniques, focusing on requests for and provisions of clarification. The data derive from an experimental study in Germany, where the face-to-face interviews were audio-recorded. A sample of 111 interviews was coded in detail. We find that conversational interviewers do make use of the ability to clarify respondent confusion. Although the technique improved response accuracy in the main study compared to SI, conversational interviewers seem to provide clarifications even when there is no evidence of respondent confusion. This may lengthen administration time and potentially increase data collection costs. Conversational interviewers also employ neutral probes, which are generally associated with standardized interviews, at an unexpectedly high rate. We conclude with suggestions for practice and directions for future research.

Using Mouse Movements to Predict Web Survey Response Difficulty

A key goal of survey interviews is to collect the highest quality data possible from respondents. In practice, however, it can be difficult to achieve this goal because respondents do not always understand particular survey questions as designers intended. Researchers have used a variety of indicators to identify and predict respondent confusion and difficulty in answering questions in different modes. In web surveys, it is possible to automatically detect response difficulty in real time. The research to date has focused on response latencies—mostly long response times—as evidence of difficulty. In addition to response latencies, however, web surveys offer rich behavioral data, which may predict respondent confusion and difficulty more directly than response times. This article focuses on one such behavior, mouse movements. We examine a set of mouse movements participants engage in when answering questions about experimental scenarios whose difficulty has been manipulated (as confirmed by respondent ratings). This approach makes it possible to determine which movements are general movements, demonstrating how a person interacts with a computer, and which movements are related to response difficulty. We find not only that certain mouse movements are highly predictive of difficulty but also that such movements add considerable value when used in conjunction with response times. The approach developed in this article may be useful in delivering help to confused respondents in real time and as a diagnostic tool to identify confusing questions.

The Effect of CATI Questions, Respondents, and Interviewers on Response Time

In this paper, we evaluate the joint effects of question, respondent, and interviewer characteristics on response time in a telephone survey. We include question features traditionally examined, such as the length of the question and format of response options, and features that have yet to be examined that are related to the layout and format of interviewer-administered questions. We examine how these question features affect the time to ask and answer survey questions and how different interviewers vary in their administration of these questions. This paper uses paradata from the Work and Leisure Today survey and uses cross-classified random effects models. Overall, most of the variation in response time is due to question characteristics, rather than respondent or interviewer attributes. Additionally, we find that question characteristics related to necessary survey design features and respondent confusion are the primary predictors of response time, with little effect of visual design features of the question. We also find modest differences in the effects of question characteristics by interviewer experience.

Evaluation of a Vision-Related Utility Instrument: The German Vision and Quality of Life Index

Multi-attribute utility instruments (MAUIs), which contain a descriptive system, including several health dimensions with associated levels of increasing severity, are used commonly to measure utilities. However, the validity of the descriptive systems rarely is examined using modern psychometric theory. Therefore, we evaluated the psychometric properties of the German version of the Vision and Quality of Life Index (VisQol), a six-item vision-related MAUI. The German VisQol was self-administered to 340 patients and 280 controls. All subjects underwent a full ophthalmologic examination, including best-corrected visual acuity (VA) testing. The psychometric properties of the VisQoL were assessed using Rasch analysis. The VisQoL's descriptive system did not function in controls. In patients, after collapsing response categories to resolve disordered thresholds and omitting misfitting persons, the measurement properties (i.e., precision, unidimensionality, and targeting) of the German VisQoL were satisfactory. Most person misfit related to unexpected responses to item 4 ("organizing assistance"). Rasch-generated person estimates were not different between age categories, sex, or underlying ocular condition, but decreased significantly with presence of visual impairment in the better eye (LogMAR ≥ 0.5, 1.20 ± 4.62 compared to 3.46 ± 3.52, P < 0.001). The German VisQoL's descriptive system displayed adequate fit to the Rasch model after removal of a large proportion of patients with poor fit statistics. However, the wording of item four should be revised to reduce respondent confusion and measurement "noise." The scale's descriptive system does not function in a sample of visually unimpaired persons, most likely due to a lack of variance in the measured trait.

Measuring Sexual Orientation and Gender Expression Among Middle-Aged and Older Women in a Cancer Screening Study

The Cancer Screening Project for Women is about the experiences of legally unmarried women aged 40 to 75 years. Prior to the implementation of a survey of experiences with breast, cervical, and colorectal cancer screenings, we used cognitive-based interviewing to evaluate questions for measuring sexual orientation and gender expression. We conducted interviews with 40 women, 19 who partner with women and 21 who partner with men. Interviews highlighted respondent confusion, clarified the meaning of terms, and improved the comprehension and utility of questions. Cognitive interview techniques can improve the validity and reliability of data collected by cancer screening programs.

The Juvenile Victimization Questionnaire: Reliability, validity, and national norms

Objective: To assess the utility and performance of the 34-item Juvenile Victimization Questionnaire (JVQ) in eliciting the recent victimization experiences of a national sample of children ages 2–17. Method: The JVQ was administered in a national random digit dial telephone survey about the experiences of 2,030 children. The experiences of children 10–17 years old were assessed through youth self-report on the JVQ, and the experiences of children 2–9 assessed through JVQ caregiver proxy report. Results: Large numbers of recent victimizations were disclosed using the JVQ (71% of the sample reporting at least one victimization in the last year, with an average of 2.63 victimizations per child). There were few indicators of respondent confusion and little resistance to even the most sensitive questions. In a test of construct validity, endorsements of JVQ items correlated well with measures of traumatic symptoms. The instrument showed adequate test-retest reliability in a 3 to 4 week re-administration. Large numbers of victimizations were reported across the spectrum of ages, and there were no major discontinuities between the self-reports and proxy reports, suggesting that caregivers provided generally adequate and comparable information to child self-reports about the experiences of children under the age of 10. Conclusion: The JVQ has potential for use in future epidemiological research as well as clinical evaluation concerning the victimization of children.

Validity of methods used to assess vitamin and mineral supplement use.

Assessing vitamin and mineral supplement use is important because supplement use per se is an exposure of interest for the risk of several chronic diseases and because supplements contribute a large proportion of total (diet plus supplement) micronutrient intake, another important exposure in epidemiologic research. Unfortunately, little is known about methods for obtaining valid information about supplement use. The authors conducted a validation study in 1996 comparing supplement data collected in a telephone interview and from a self-administered questionnaire with data derived from a detailed, in-person interview and transcription of the labels of supplement bottles (i.e., a gold standard) among adult supplement users in Washington State (n = 104). Spearman correlation coefficients comparing average daily supplemental vitamin and mineral intake from the interview or questionnaire with the gold standard ranged from 0.76 (95% confidence interval 0.66-0.83) for vitamin C to 0.08 (95% confidence interval -0.14 to 0.29) for iron, with a mean of about 0.5. The principal sources of error were inaccurate assumptions about the micronutrient composition of multiple vitamins and respondent confusion regarding the distinction between multiple vitamins and single supplements. These results suggest that commonly used epidemiologic methods of assessing supplement use may incorporate significant amounts of error in estimates of some nutrients.

Information Processing Capacity and Attitude Measurement

Green and Rao, using the criterion of information recovery and simulated data, concluded that sevenpoint scales were best [3]. Benson, pointing out that their criterion was not always applicable, suggested that wherever only averages are required, dichotomous or trichotomous scales suffice [1]. Jacoby and Matell reached similar conclusions using summary rating indices [9]. Lehmann and Hulbert's simulation concluded that a minimum of five categories was desirable unless there was averaging, either across individuals or scales [10]. Most research on the number of scale categories has focused on the desirable lower limit. At the upper limit, there are possible costs of respondent confusion and increased questionnaire length associated with increasing the number of categories [7]. Apart from this type of measurement error, however, Miller has found that over a wide range of unidimensional sensory variables, humans are incapable of discriminating among more than about 10 discrete categories. He concluded that this resulted from limited information processing capacity which appeared reasonably invariant over a number of different sensory attributes [11]. If this phenomenon were to apply to such common marketing measurement problems as rating stimuli on dimensions, then it would raise legitimate