Measuring Change in Employment Characteristics: The Effects of Dependent Interviewing

Abstract

We have demonstrated that the use of PDI results in lower levels of observed change in each of a number of characteristics of the respondent's employment. The reduction in observed change is particularly pronounced for SOC, SIC, and the number of employees at the respondent's workplace. This reduction in observed change appears to represent a reduction in measurement error as the effect of PDI is particularly pronounced amongst respondents who have not reported a change in job between survey waves. Levels of change in employment characteristics amongst INDI respondents who have not reported a change in job remain implausibly high. Furthermore, it seems that the reduction in measurement error brought about by PDI is particularly associated with certain employment characteristics. Finally, we found that measurement error was particularly reduced amongst respondents aged 36 or over and amongst the most highly qualified respondents. In attempting to summarize the nature of the characteristics, both employment and demographic, associated with a propensity for PDI to reduce measurement error in estimates of change, we would suggest that many of these characteristics may be associated with an increased propensity to have complex jobs, the characteristics of which are difficult to describe, or are associated with increased uncertainty on the part of the respondent regarding the information being sought. Complex jobs are likely to be more common amongst more highly qualified people, those in managerial, administrative or professional occupations and those working at large workplaces and in the private sector. These are the groups amongst whom PDI reduced the observed levels of change in SOC and SIC. The more difficult it is to describe a particular characteristic of a job, the more likely it is that it will be described inconsistently in two survey interviews some time apart. This leads to measurement error in estimates of change. Similarly, number of employees at the workplace may be something that respondents are less likely to know accurately if the workplace is large and if they are in the public sector (where workplaces may be larger on average, and also more impersonal). Thus, perhaps what we are observing here is simply that survey respondents who are more prone to measurement error with traditional independent interviewing will be more likely to have reduced measurement error if interviewed using PDI. An alternative explanation might be that PDI reduces the error disproportionately across sample subgroups due to differences in the way the change in interviewing method affects the cognitive processes during the interview. We find no obvious support for this. For example, if this explanation held we might expect to find a greater reduction in measurement error amongst respondents with lower levels of qualifications. We find the opposite. However, we would also note that such an effect might be difficult to identify due to the association of qualifications with employment characteristics. For example, people with lower qualifications may indeed be prone to greater levels of measurement error other things being equal, only for this effect to be outweighed by the greater tendency of people with lower qualifications to be in occupations that are easy to describe and easy for coders to recognize - that is occupations that are prone to lower levels of measurement error. We conclude that PDI is an effective questioning method for the reduction of measurement error associated with measures of change in employment characteristics. Additionally, measurement error in measures of change from traditional independently-collected interviews is likely to be greater amongst occupations that are difficult to describe consistently. This is likely to include jobs where employees have a range of varied tasks, as opposed to routine jobs involving a single task, or a limited number of closely-related tasks. Data analysts might be advised to allow for this possibility. © The Author 2006. Published by Oxford University Press on behalf of The World Association for Public Opinion Research. All rights reserved.