Software Engineering Measurement Research Articles

Review of "Software engineering measurement" by John C. Munson. Auerbach Publications 2003.

Review of "Software engineering measurement" by John C. Munson. Auerbach Publications 2003.

A ranking of software engineering measures based on expert opinion

This research proposes a framework based on expert opinion elicitation, developed to select the software engineering measures which are the best software reliability indicators. The current research is based on the top 30 measures identified in an earlier study conducted by Lawrence Livermore National Laboratory. A set of ranking criteria and their levels were identified. The score of each measure for each ranking criterion was elicited through expert opinion and then aggregated into a single score using multiattribute utility theory. The basic aggregation scheme selected was a linear additive scheme. A comprehensive sensitivity analysis was carried out. The sensitivity analysis included: variation of the ranking criteria levels, variation of the weights, variation of the aggregation schemes. The top-ranked measures were identified. Use of these measures in each software development phase can lead to a more reliable quantitative prediction of software reliability.

A hybrid approach to analyze empirical software engineering data and its application to predict module fault-proneness in maintenance

Knowledge discovery from software engineering measurement data is essential in deriving the right conclusions from experiments. Various data analysis techniques may provide data analysts with different and complementary insights into the studied phenomena. In this paper, two data analysis techniques – Rough Sets (RSs) and Logistic Regression (LR) are compared, from both the theoretical and the experimental point of view. In particular, the empirical study was performed as a part of the ESPRIT/ESSI project CEMP on a real-life maintenance project, the DATATRIEVE™ project carried out at Digital Engineering Italy. We have applied both techniques to the same data set. The goal of the experimental study was to predict module fault-proneness and to determine the major factors affecting software reliability in the application context. The results obtained with either analysis technique are discussed and compared. Then, a hybrid approach is built, by integrating different and complementary knowledge obtained from either approach on the fault-proneness of modules. This knowledge can be reused in the organizational framework of a company-wide experience factory.

Requirements for the knowledge-based support of software engineering measurement plans

In order to improve the quality of software systems, measurement programs have been implemented in many companies to support process improvement activities. The planning and implementation of a successful measurement program requires, in practice, a significant amount of effect. Cost may be reduced and quality of measurement may be improved by providing knowledge-based support and reusing experiences gathered on past measurement programs. In this article, we state the requirements for the knowledge-based support of planning measurement programs based on the Goal/Question/Metric paradigm. Reuse opportunities are precisely identified, the knowledge to be captured for effective reuse is identified and structured, and reuse scenarios are provided.

Reply to: "Property-based software engineering measurement"

L.C. Briand, S. Morasca and V.R. Basili (ibid., vol. 22, no. 1, pp. 68-85, Jan. 1996) introduced a measurement-theoretic approach to software measurement and criticized (among others) the work of the author, but they misinterpreted his work. The author does not require additive software (complexity) measures as Briand, Morasca and Basili state. The author uses the concept of the extensive structure in order to show the empirical properties behind software measures. Briand, Morasca and Basili use the concept of meaningfulness in order to describe scales and that certain scale levels are not excluded by the Weyuker properties. However, they do not consider that scales and scale types are different things.

Comments on "Property-based software engineering measurement: refining the additivity properties"

The measure property set of Briand, Morasca, and Basili (1996) establishes the foundation of a real software measurement theory. Unfortunately, a number of inconsistencies related to additivity properties might hinder its acceptance and further elaboration. The authors show how to remove the ambiguity in the property definitions.

Response To: Comments On "property-based Software Engineering Measurement: Refining The Additivity Properties"

Response To: Comments On "property-based Software Engineering Measurement: Refining The Additivity Properties"

Property-based software engineering measurement

Little theory exists in the field of software system measurement. Concepts such as complexity, coupling, cohesion or even size are very often subject to interpretation and appear to have inconsistent definitions in the literature. As a consequence, there is little guidance provided to the analyst attempting to define proper measures for specific problems. Many controversies in the literature are simply misunderstandings and stem from the fact that some people talk about different measurement concepts under the same label (complexity is the most common case). There is a need to define unambiguously the most important measurement concepts used in the measurement of software products. One way of doing so is to define precisely what mathematical properties characterize these concepts, regardless of the specific software artifacts to which these concepts are applied. Such a mathematical framework could generate a consensus in the software engineering community and provide a means for better communication among researchers, better guidelines for analysts, and better evaluation methods for commercial static analyzers for practitioners. We propose a mathematical framework which is generic, because it is not specific to any particular software artifact, and rigorous, because it is based on precise mathematical concepts. We use this framework to propose definitions of several important measurement concepts (size, length, complexity, cohesion, coupling). It does not intend to be complete or fully objective; other frameworks could have been proposed and different choices could have been made. However, we believe that the formalisms and properties we introduce are convenient and intuitive. This framework contributes constructively to a firmer theoretical ground of software measurement.

Measuring the readability and maintainability of hyperdocuments

AbstractThe problem of poorly written hyperdocuments has already been identified. Furthermore, there is no complete definition of hyperdocument quality and the methodology and tools that will help in analysing and assessing the quality of hyperdocuments are missing. The ability to measure attributes of hyperdocuments is indispensable for the fields of hyperdocument authoring and hypertext engineering. Useful paradigms can be drawn from the practices used in the software engineering and software measurement fields.In this paper we define a hyperdocument quality model, based on the ideas of the well‐known Factor‐Criteria‐Metric hierarchical model. The important factors of readability and maintainability are defined, as well as the corresponding criteria. Finally, structure metrics, that can be computed on the hypertext graph, are presented. Most of these metrics are derived from well‐known software metrics.Experimentation is a key issue for the application of measurement, and flexible tools for the automatic collection of measures are needed to support it. Athena, a tool that was originally developed for software measurement and later tailored to meet hypertext measurement needs, is used for hyperdocument measurement.

A taxonomy for combining software engineering and human-computer interaction measurement approaches: towards a common framework

The rapid development of any field of knowledge brings with it unavoidable fragmentation and proliferation of new disciplines. The development of computer science is no exception. Software engineering (SE) and human-computer interaction (HCI) are both relatively new disciplines of computer science. Furthermore, as both names suggest, they each have strong connections with other subjects. SE is concerned with methods and tools for general software development based on engineering principles. This discipline has its roots not only in computer science but also in a number of traditional engineering disciplines. HCI is concerned with methods and tools for the development of human-computer interfaces, assessing the usability of computer systems and with broader issues about how people interact with computers. It is based on theories about how humans process information and interact with computers, other objects and other people in the organizational and social contexts in which computers are used. HCI draws on knowledge and skills from psychology, anthropology and sociology in addition to computer science. Both disciplines need ways of measuring how well their products and development processes fulfil their intended requirements. Traditionally, SE has been concerned with "how software is constructed" and HCI with "how people use software". Given the different histories of the disciplines and their different objectives, it is not surprising that they take different approaches to measurement, Thus, each has its own distinct "measurement culture". In this paper we analyse the differences and the commonalities of the two cultures by examining the measurement approaches used by each. We then argue the need for a common measurement taxonomy and framework, which is derived from our analyses of the two disciplines. Next we demonstrate the usefulness of the taxonomy and framework via specific example studies drawn from our own work and that of others and show that, in fact, the two disciplines have many important similarities as well as differences and that there is some evidence to suggest that they are growing closer. Finally, we discuss the role of the taxonomy as a framework to support: reuse, planning future studies, guiding practice and facilitating communication between the two disciplines.

METKIT: training in how to use measurement as a software management tool

A Science is as mature as its measurement tools. Louis Pasteur The main purpose of this paper is to present the results of the ESPRIT project METKIT, a three-year collaborative technology transfer project in the field of software engineering measurement. METKIT was concerned with developing a set of integrated training courses that show how to implement process improvement programmes and use measurement as a powerful management tool. METKIT can be used by an organization to enable them to provide in-house training to their managers and software developers to enable them to understand control and then improve software development.

Applications of information theory to software engineering measurement

In the 1990s, there is an emphasis on finding ways to lower software cost and improve quality. Thus, it is very important to quantify and measure factors, such as software complexity, which have been shown to affect cost and quality. Researchers have defined many software complexity measures, and have exploited them to identify fault-prone program modules, to predict the number of faults that testing and operation will reveal, or to assess maintainability. Information theory based software measures are attractive because they quantify, with a standard unit, the amount of information in an abstraction of a program. The unit of measure is a bit. The purpose of this paper is to survey the state of the art of applications of information theory to software measurement, beginning in 1972. Information theory based measures have been applied to most phases of the development lifecycle. However, there have been relatively few papers. Most measures have not been empirically validated. One can conclude that the field is in its infancy. Relevant concepts of information theory are briefly described, and tables summarize the references from various perspectives. Since the amount of research, thus far, has been very limited, researchers will find numerous opportunities to validate, refine, and improve the measures presented here. Such research should give future practitioners useful software measures for each phase of the lifecycle.

Software metrics and measurement principles

Software measurement is widely advocated as a fundamental constituent of an engineering approach to planning and controlling software development. Unfortunately, there is a dichotomy between the quantity of developed metrics and those used. This paper provides a tutorial review of software engineering measurement indicating the depth and breadth of the field. Individual metrics are not described due to the interest of this paper being on the measurement process and not the products of that process. Generic problems have been identified within existing measurement processes, these provide learning points for the expression of measurement principles. These principles are classified and described according to their position within the formulation, analysis and application stages of measurement. Conclusions are elaborated that suggest that existing measurement frameworks for applying measurement - often called measurement methods - do not provide sufficient support for the principles and their continued use will only serve to replicate the problems. In order to improve the products i.e. metrics, the measurement process requires improvement through inclusion of these principles in a new method.

On statistics in software engineering measurement

This paper is a brief account of contemporary statistical activity in software engineering measurement, with particular reference to the CEC ESPRIT, programme. The report is intended to be of interest to technical and non-technical managerials staff and assumes little prior knowledge of statistics. It is based on work done for the COSMOS project.1

Integrating test metrics within a software engineering measurement program at Eastman Kodak Company: A follow-up case study

This article provides an overview of an initial measurement program implemented within a software engineering organization at Eastman Kodak Company. It details the process used to integrate additional measures into the existing measurement process. The new measures are the result of development and measurement process evolution. The article describes the benefits, successes, cautions, and failures encountered during the evolution of this measurement program.

The METKIT Academic package

The METKIT Academic Package provides materials for teaching undergraduate and postgraduate academic students about the use of measurement in software engineering. The materials have been designed for maximum flexibility and ease of use. In the paper the structure and content of the materials are briefly explained.

The METKIT CAI system

The METKIT Computer Aided Instruction (CAI) System primarily addresses software engineers. It supports them directly at their workplace by providing knowledge about the concepts of software engineering measurement on their workstation. The system integrates concepts of hypertext with those of computer aided instruction, providing different types of access to conceptual knowledge about software measurement. Thus, both software engineers with some experience in software measurement and beginners in that domain are adequately supported. The first type of use may freely browse through conceptual knowledge whereas the second will find guidance through that knowledge based on particular learning objectives and didactic strategies.

Need for well-balanced education in software engineering measurement

This paper deals with a perceived need for a well-balanced education (or in some cases re-education) in the subject of software engineering measurement, more commonly although less accurately known as software metrics. This need was confirmed by a major survey of both industry and academia conducted in 1989.

Application of spreadsheet software in software engineering measurement technology

The application of measurement technology to software engineering and science is relatively new and therefore the benefits of using it are not recognised enough. The development and use of adequate measures of software and its development process are essential, however, to the production of cost-effective and reliable software. The main objective of the paper is to show that spreadsheet software, hybrid metrics, and metrics life-cycle models can improve the situation.

Project management in the development of scientific software

This contribution is a rough outline of a comprehensive project management model for the development of software for scientific applications. The model was tested in the unique environment of the Siemens AG Corporate Research and Technology Division. Its focal points are • the structuring of project content - the so-called phase organization, • the project organization and • the planning model used, and its particular applicability to innovative projects. The outline focuses largely on actual project management aspects rather than associated software engineering measures.