Improving management information systems research: question order effects in surveys

Abstract

This paper demonstrates the impact that question order can have in using survey research in management information systems (MIS). Using data from a survey of 379 companies concerning electronic data interchange (EDI), significant differences are found in the evaluation of successful implementations. The overall evaluation of success is increased (or decreased) depending upon alternative versions of question order. In addition, the analyses show that the main effects from question order interact in different ways with respondent characteristics, including functional area, educational background and being a systems user. The bias tends to be consistent for some types of respondents, such as systems users. However, other classes of respondents, especially persons with professional backgrounds in systems or MIS, are influenced in more complex ways by question order. The order in which queries regarding systems output and quality occur tends to be especially important in anchoring subsequent responses and overall assessments. The final section addresses corrective actions and future research.