Intervention Effect Rates as a Path to Research Relevance: Information Systems Security Example

Abstract

In the current information systems security (ISS) research, new theory contributions are especially valued. This research typically reflects the following formula: Suggest a new theory (or set of constructs) of ISS and show that it is empirically supported, then suggest another new theory (or set of constructs with some linkages) and show that it is empirically supported, and so on. Despite the merits of this approach, it leaves out many important scientific aspects. For example, after more than 30 years of ISS research, (1) we know little about the conditions and situations to which new theories (or constructs) do not apply; (2) we do not know which new theories are more effective than others in solving an ISS problem; and (3) we have not demonstrated that our best research, or new theoretical contributions, can beat industry best practices or practitioners’ intuitive approaches. We suggest that ISS research be examined in terms of long-term research programs comprising four levels: metalevel research, basic research, applied research, and postintervention research. The ultimate success of such programs does not entail new theories, “contextualized theories,” or adding IT artifacts to theories; rather, it hinges on the question of which program can demonstrate the best intervention effect rate for a given ISS problem. The lack of demonstrated intervention effectiveness (e.g., by showing treatment effect rates) is one important inhibitor that may prevent ISS research from achieving relevance in practice. Without reporting such evidence, ISS research cannot overpower the folklore, fads, or industry “best practices” that often guide operations. With such treatment effect rates, evidence-based practice may become more justifiable. We believe that our ideas also can be applied to information systems research in general.