Increasing anomaly handling efficiency in large organizations using applied machine learning

Abstract

Maintenance costs can be substantial for large organizations (several hundreds of programmers) with very large and complex software systems. By large we mean lines of code in the range of hundreds of thousands or millions. Our research objective is to improve the process of handling anomaly reports for large organizations. Specifically, we are addressing the problem of the manual, laborious and time consuming process of assigning anomaly reports to the correct design teams and the related issue of localizing faults in the system architecture. In large organizations, with complex systems, this is particularly problematic because the receiver of an anomaly report may not have detailed knowledge of the whole system. As a consequence, anomaly reports may be assigned to the wrong team in the organization, causing delays and unnecessary work. We have so far developed two machine learning prototypes to validate our approach. The latest, a re-implementation and extension, of the first is being evaluated on four large systems at Ericsson AB. Our main goal is to investigate how large software development organizations can significantly improve development efficiency by replacing manual anomaly report assignment and fault localization with machine learning techniques. Our approach focuses on training machine learning systems on anomaly report databases; this is in contrast to many other approaches that are based on test case execution combined with program sampling and/or source code analysis.