Bug Patterns in Probabilistic Programming Systems

Abstract

Probabilistic programming systems allow developers to model random phenomena and perform reasoning about the model efficiently. As the number of probabilistic programming systems is growing significantly and are used more and more widely, the reliability of such systems is becoming very important. It is crucial to analyze real bugs of existing similar systems in order to develop efficient bug detection tools for probabilistic programming systems. This paper conducts an empirical study investigating bugs and their features on PyMC3, a real probabilistic programming system. Among 271 closed bugs, we identified 20 bugs that are unique to probabilistic programming languages and extracted eight bug patterns from these bugs. The result showed that many of the bugs were caused by types. We also propose some possible methods for automatically detecting these bug patterns. It is expected that this will contribute to the development of bug detection tools by capturing the characteristics of bugs in actual probabilistic programs in the future.