Empirical Analysis of Hidden Technical Debt Patterns in Machine Learning Software

Abstract

[Context/Background] Machine Learning (ML) software has special ability for increasing technical debt due to ML-specific issues besides having all the problems of regular code. The term “Hidden Technical Debt” (HTD) was coined by Sculley et al. to address maintainability issues in ML software as an analogy to technical debt in traditional software. [Goal] The aim of this paper is to empirically analyse how HTD patterns emerge during the early development phase of ML software, namely the prototyping phase. [Method] Therefore, we conducted a case study with subject systems as ML models planned to be integrated into the software system owned by Vasttrafik, the public transportation agency in the west area of Sweden. [Results] During our case study, we could detect HTD patterns, which have the potential to emerge in ML prototypes, except for “Legacy Features”, “Correlated features”, and “Plain Old Data Type Smell”. [Conclusion] Preliminary results indicate that emergence of significant amount of HTD patterns can occur during prototyping phase. However, generalizability of our results require analyses of further ML systems from various domains.