Identifying Productivity Drivers by Modeling Work Units Using Partial Data

Abstract

We describe a new algorithm for estimating a model for an independent variable that is not directly observed but that represents one set of marginal totals of a sparse nonnegative two-way table whose other margin and zero pattern are known. The application that inspired the development of this algorithm arises in software engineering. We seek to identify those factors that affect the effort required for a developer to make a change to the software—for instance, to identify difficult areas of the code, measure changes in the code difficulty through time, and evaluate the effectiveness of development tools. Unfortunately, measurements of effort for changes are not available in historical data. We model change effort using a developer's total monthly effort and information about which changes he/she investigated in each month. We illustrate a few specific applications of our tool, demonstrate that the algorithm is an instance of the EM algorithm, and present a simulation study that speaks well of the reliability of the results our algorithm produces. In short, this algorithm allows analysts to quantify the impact of a promising software engineering tool or practice on coding effort.