Abstract
High Energy Physics (HEP) software environments often involve ∼ hundreds of external packages and libraries, and similar numbers of experiment-specific, science-critical packages—many under continuous development. Managing coherent releases of the external software stack is challenging enough, but managing the highly-collaborative—and distributed—development of a large body of code against such a stack adds even more complexity and room for error. Spack is a popular Python-based package management tool with a specific focus on the needs of High Performance Computing (HPC) systems and system administrators whose strength is orchestrating the discrete download, build, testing, and installation of pre-packaged or tagged third-party software against similarly stable dependencies. As such it is becoming increasingly popular within HEP as that community makes increasing use of HPC facilities, and as efforts to develop future HPC systems utilize Spack to provide scientific software on those platforms [1]. SpackDev is a system to facilitate the simultaneous development of interconnected sets of packages. Intended to handle packages without restriction to one internal build system, SpackDev is integrated with Spack as a command extension in order to leverage features such as dependency calculations and build system configuration, and is generally applicable outside HEP. We describe SpackDev’s features and development over the last two years, initial experience using SpackDev in the context of the LArSoft liquid argon detector toolkit, and work remaining before it can be considered a fully-functional multi-package build system for HEP experiments utilizing Spack.
Highlights
Development of scientific software is by nature a collaborative exercise: scientists working together on a paper or project will divide between them the labor of writing code to produce, simulate or analyze data
In addition to the framework, we provide an optional single-package CMake [9]-based build system, cetbuildtools [10], and a multi-package development system we developed, MRB, both of which depend upon specific behaviors of UPS [7, 8] to integrate locally-developed packages with a centrally-installed release of the full software suite
Packages Spack knows how to build are represented by recipes, implemented as Python classes inheriting from a base class Package or a subclass thereof representing a known build system such as class CMakePackage, class MakefilePackage or class AutotoolsPackage
Summary
Development of scientific software is by nature a collaborative exercise: scientists working together on a paper or project will divide between them the labor of writing code to produce, simulate or analyze data. Code for HEP experiments in particular is generally very library- and plugin-oriented: frameworks connect data sources, algorithms, and outputs. From a software engineering perspective, the HEP environment is complex: simulations, reconstruction algorithms and analyses change as an experiment improves its understanding of its experimental hardware, data acquisition systems, and the underlying physics being studied. Where a project utilizes a significant amount of C++ in internal or external software, compatibility between the compiled code of different libraries used in the same application—the “Application Binary Interface,” (ABI)—is especially critical and requires careful management of the development environment to ensure compilations against pre-installed software produce consistent builds of composite applications. In HEP, this problem is often solved with a multi-package development system or workflow
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
