THE RELIABILITY OF DIFFERENCE SCORES: A PRELIMINARY INVESTIGATION OF A NEED DEFICIENCY SATISFACTION SCALE.

Abstract

A fundamental criterion of sound research is that measuring instruments are reliable, i.e. free of measurement error. Cronbach, Gleser, Nanda, and Rajaratnam [1] have recently defined the problem of reliability as a question of generalizability, i.e., the accuracy with which an investigator can generalize from a sample of measurement results to a universe of measurement results. In addition, they have defined a variety of universes to which one may wish to generalize, including the universe of items sampled in a measure, occasions of administration, and observers. This study is a preliminary investigation of the degree to which items sampled in the Porter Need Satisfaction Questionnaire generalize to a domain of difference score items assessing job satisfaction. A review of the job satisfaction research literature suggests that the reliability or generalizability of measures derived from need deficiency models has not been widely considered. Evans [3], Schwab and Cummings [15], Wanous and Lawler [16], and Imparato [8] have investigated a variety of conceptual problems in need deficiency measures of satisfaction but have not raised the issue of reliability. One exception is a recent paper by Oleno and Zedeck [13], who included unreliability in their discussions of difficulties associated with difference score measures of satisfaction. Need deficiency satisfaction measures are based on absolute differences between responses to two questions: (1) how much is there and (2) how much should there be of several sources of employee satisfaction. [14] Many authors including Gulliksen [51, Guilford [4], McNemar [11], Lord [9], Webster and Bereiter [17], Cronbach and Furby [2] and Cronbach, et al. [1] have demonstrated a multitude of measurement problems created by the difference-score approach, including that of unreliability. This study focuses exclusively on the problem of estimating the reliability or generalizability of a modified Porter [14] scale, a satisfaction scale based upon the need deficiency model, in two managerial and two nonmanagerial samples.