F-metric: a WWW-based framework for intelligent formulation and analysis of metric queries

Abstract

As an organization matures, quantitative techniques are employed to make software project management more systematic, informed and under control. This is typically done through the collection and analysis of software metrics to measure the performance of projects. While many organizations utilize software metrics to analyze project issues and answer management queries, the manner in which such queries are answered varies from organization to organization. As the number of metrics grows, interpretation of raw metrics in the context of management goals becomes difficult. The method and tool that we developed attempts to bridge the gap between the project managers' mental model of a project and raw software metrics. Based on industrial surveys, we identified types of queries about the project progress, resources and schedule that project managers often ask during project execution. Next, we built a conceptual model for project metrics relevant to those queries, justifying our selection of a core set of test and evaluation metrics. We defined a project management query language (PMQL for short), based on concepts of an extended Entity-Relationship model, which we used to formulate project queries at a conceptual level. Finally, we present a flexible WWW-based framework which can be used by experts to formulate and update management queries in PMQL, modify heuristics based on which queries are evaluated, as well as to dynamically display solutions to existing queries. The flexibility of our framework means that new queries may be added, heuristics to queries may be changed, and display of results can be changed on-the-fly via mutual cooperation between managers and technical experts of the organization without any changes to code.