A Family of Experiments to Compare Two Model-Driven Development Tools vs a Traditional Development Method

Abstract

<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Context:</b> There are many papers which extol the benefits of Model-Driven Development (MDD) compared to traditional developments. However, the adoption of MDD to develop fully functional systems without coding is not frequent. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Objective:</b> This paper presents a family of experiments with 7 replications and 56 sample units to compare a traditional method versus MDD, analysing two MDD tools. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Method:</b> The factor in the experiment is the method with two treatments (traditional and MDD). We analyse together all replications thanks to two moderator variables in a multilevel hierarchy: Replication (from 1 to 7) and MDD Tool (INTEGRANOVA and WebRatio). Response variables are Functional Suitability, measured in terms of effectiveness in simple and complex problems; Effort, measured as time to develop simple problems; and Satisfaction, measured in terms of perceived ease of use, perceived usefulness and intention to use. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Result:</b> Functional Suitability in MDD yields significantly better results both in simple and complex problems, while Effort for simple problems is also significantly better for MDD. Differences for Functional Suitability in complex problems are greater than with simple ones. Considering the MDD tools, INTEGRANOVA yields significantly better Functional Suitability in complex problems. Regarding Satisfaction, replications with WebRatio have a better intention to use than replications with INTEGRANOVA. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusions:</b> Even though MDD yields better Functional Suitability and Effort, the subjects’ Satisfaction with MDD is not better than with a traditional method.