Abstract
Software is an essential and rapidly evolving component of modern high energy physics research. The ability to be agile and take advantage of new and updated packages from the wider data science community is allowing physicists to efficiently utilise the data available to them. However, these packages often introduce complex dependency chains and evolve rapidly introducing specific, and sometimes conflicting, version requirements which can make managing environments challenging. Additionally, there is a need to replicate old environments when generating simulated data and to utilise pre-existing datasets. Nix is a “purely functional package manager” which allows for software to be built and distributed with fully specified dependencies, making packages independent from those available on the host. Builds are reproducible and multiple versions/configurations of each package can coexist with the build configuration of each perfectly preserved. Here we will give an overview of Nix followed by the work that has been done to use Nix in LHCb and the advantages and challenges that this brings.
Highlights
Intensive areas of modern research, such as high energy physics, provide unique challenges for software packaging
Software is used at a massive scale for processing large datasets using heterogeneous resources, such as the Woldwide Large Hadron Collider (LHC) Computing Grid [1]
Nix is an ideal candidate for this task and there is interest from the wider high performance computing (HPC) community in Nix, with several organisations working to improve relevant parts of Nix such as support for InfiniBand networking, the Intel Math Kernel Library and the Intel Compiler Collection
Summary
Intensive areas of modern research, such as high energy physics, provide unique challenges for software packaging. Installed packages are kept in a unique subdirectory of the store, typically /nix/store/ This subdirectory is named using a cryptographically secure hash of all inputs to the build, including the build sources, configuration and dependencies to allow for an unlimited number of versions and configurations to be available simultaneously, without any risk of conflicts between installations. Source files, such as tarballs and patches, are defined using a hash of their content. One of the disadvantages of building deep stacks is that it is time consuming and inconvenient for many use cases To mitigate this issue Nix can query static web servers using the package’s hash to download a signed tarball of the build products. Binaries can be served directly, or uploaded using a plugin system (most commonly to a S3 compatible endpoint)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
