Simulating Software Evolution to Evaluate the Reliability of Early Decision-making among Design Alternatives toward Maintainability

Abstract

Critical decisions among design altern seventh atives with regards to maintainability arise early in the software design cycle. Existing comparison models relayed on the structural evolution of the used design patterns are suitable to support such decisions. However, their effectiveness on predicting maintenance effort is usually verified on a limited number of case studies under heterogeneous metrics. In this article, a multi-variable simulation model for validating the decision-making reliability of the derived formal comparison models for the significant designing problem of recursive hierarchies of part-whole aggregations, proposed in our prior work, is introduced. In the absence of a strict validation, the simulation model has been thoroughly calibrated concerning its decision-making precision based on empirical distributions from time-series analysis, approximating the highly uncertain nature of actual maintenance process. The decision reliability of the formal models has been statistically validated on a sample of 1,000 instances of design attributes representing the entire design space of the problem. Despite the limited accuracy of measurements, the results show that the models demonstrate an increasing reliability in a long-term perspective, even under assumptions of high variability. Thus, the modeling theory discussed in our prior work delivers reliable models that significantly reduce decision-risk and relevant maintenance cost.