Special Theme of Research in Information Systems Analysis and Design-IV Evaluation Criteria for Information Systems Development Methodologies

Abstract

The evaluation of information systems development methodologies is becoming increasingly important. Some researchers propose their own criteria for conceptual evaluations. But such criteria are often constrained by the limitation of the researcher’s view toward and experience with development methodologies. Furthermore, existing evaluation criteria are either not practical for direct measurement or not tested for validity and reliability. The lack of a generally acceptable, practically valid, and reliable set of criteria for evaluation hinders the development of knowledge in this area. Our study is a step toward developing a systematic process to evaluate information systems development methodologies. We captured the opinions of a group of twenty-eight researchers and practitioners who are experienced in information systems development methodologies. Through a systematic content analysis, the authors classified these criteria into three categories: (1) methodology design, (2) methodology use, and (3) methodology deliverables. The three categories of criteria are not totally separate in that they represent the means and goals of information systems development methodologies. Appropriate criteria should be applied depending on the stages when evaluation is performed.