Abstract
Despite general agreement regarding the usefulness of statistical process control (SPC) tools for monitoring paradata, using SPC from an early phase of the survey fieldwork is rather rare. This study focuses on one type of paradata—interview duration—to fill this void. First, we establish a procedure based on the idea of enabling fieldwork monitoring for the seventh round of the European Social Survey in Belgium from its start. The impact of respondent characteristics on interview duration is controlled for by multiple regression. Moreover, we simulate the real conditions of an ongoing survey data collection process by cumulating data and repeating the identification of problematic interviews each week, on the basis that “new” data are available. Second, for each interview we record and track the results with regard to whether or not it is problematic over the fieldwork period, to examine the consistency of our findings. We find that as more data becomes available, the results concerning whether an interview is problematic changes in only 0.3% of the cases. Out of the 27 interviews identified as problematic when all information was available, 25 were immediately identified once relevant information was available. Overall, these findings suggest that SPC tools are reliable and efficient in a survey context, and accordingly have great potential for allowing survey practitioners to focus on the interviews for which further examination is needed immediately, rather than when the data collection has been completed.
Highlights
For quite some time, paradata has been popular in the evaluation of data quality in a total survey error framework
Considering the implications interview duration has for data quality, we address the research gap concerning the timeliness and effectiveness of the application of control charts when face-to-face interviews are used by investigating two research questions: Research Question 1 (RQ1): How can control charts be used to monitor interview duration from the very beginning of the data collection period? Research Question 2 (RQ2): Do the interviews identified as problematic at an early stage of fieldwork hold at a later phase of the fieldwork?
Older respondents take more time to answer questions, the interview duration increases when respondents frequently ask for clarification, having fewer applicable questions reduce the interview duration, and the order in which interviews are taken has a negative effect on the duration
Summary
Paradata has been popular in the evaluation of data quality in a total survey error framework (for detailed overviews, see Kreuter & Olson, 2013; Olson & Parkhurst, 2013). A statistical process control (SPC) framework is a promising tool to analyze paradata during survey data collection (Kreuter, Couper, & Lyberg, 2010). The key tools of SPC, use control limits to monitor the performance of a process over time to determine whether special variation exists in the process. If the paradata estimates are plotted within the control limits, the process is seen as “in control,” if not, it is considered as “out of control.”. In the latter case, the potentially problematic interviews and interviewers with extreme values on the key paradata estimates can be identified and further followed during survey data collection If the paradata estimates are plotted within the control limits, the process is seen as “in control,” if not, it is considered as “out of control.” In the latter case, the potentially problematic interviews and interviewers with extreme values on the key paradata estimates can be identified and further followed during survey data collection
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
