WatchMan project—A Python CASE framework for High Energy Physics data analysis in the LHC era

Abstract

The world's largest particle collider LHC is taking data at CERN, in Geneva, providing a huge amount of data to be looked at, of the order of several Petabytes per year. Nowadays, Data Analysis in High Energy Physics (HEP) means handling billions of experimental data in custom software frameworks. Physicists have to access and select data interacting with the experiment using dedicated tools; they also have to apply filter functions and analysis algorithms to test hypotheses about the physics underlain. Modern HEP experiments rely on complex software frameworks, hence writing the analysis code is not always an easy task, and the learning curve is usually quite steep. Moreover each hypothesis requires a dedicated analysis, in order to have a better control on it and to be able to validate the results among different groups of researchers. And the writing of so many analyses can be error prone and time consuming.In order to ease the writing of such data analysis code, we built a software-generator: the idea is that the user inserts the settings of the physics analyses, and the final analysis code is automatically and dynamically generated, ready to be run on data. Python helped us to build such a package. Its high-level and dynamic nature, together with its flexibility and prototyping speed are the key features which made our choice.So we conceived and developed WatchMan, a Python CASE (Computer-Aided Software Engineering) framework to automatically generate reliable, easy to maintain and easy to validate HEP data analysis code.