The measurement of computer literacy: a comparison of self-appraisal and objective tests

Abstract

Whenever decisions are made based upon a person's level of computer literacy, it is important that such expertise is accurately assessed. This paper takes a thorough methodological approach to the measurement of computer literacy using both objective and self-appraisal tests. While objective tests have been used on many occasions to measure computer literacy, they suffer from generalizability problems. Self-appraisal tests, on the other hand, are subject to leniency bias by the respondents. Taken together, though, the potential exists for the establishment of a computer literacy assessment instrument with high levels of generalizability and accuracy. For this research, an objective test for computer literacy was developed and an existing self-appraisal test was extended for use in a computer literacy assessment experiment. It was found that the self-appraisal test is a more lenient performance indicator than the objective test. Both male and female subjects exhibited substantial self-leniency in their self-appraisals, but both self-leniency and gender-based differences in self-appraisal decreased as the subjects' level of computer expertise increased. Finally, the low level of convergence between the self-appraisal test and the objective test found in this study cast doubt on the ability of any self-appraisal test to assess accurately computer literacy by itself. A combination of different measures may be more appropriate when it is important to determine computer literacy levels accurately.