Abstract
The Thomas Jefferson National Accelerator Facility (JLab) 12GeV accelerator upgrade completed in 2015 is now producing data at volumes unprecedented for the lab. The resources required to process this data now exceed the capacity of the onsite farm necessitating the use of offsite computing resources for the first time in the history of JLab. GlueX is now utilizing NERSC and PSC for raw data production. Details of the workflow are presented.
Highlights
The Thomas Jefferson National Accelerator Facility (JLab) 12GeV accelerator upgrade completed in 2015 is producing data at volumes unprecedented for the lab
This motivates us to use a system that can distribute newly compiled versions which can be run on all production platforms, regardless of the host OS. This is done using a Docker container to provide crossplatform uniformity and the CERN Virtual Machine File System (CVMFS)[3] for binary file distributions. Both National Energy Research Scientific Computing Center (NERSC)[4] and Pittsburgh SuperComputing Center (PSC)[5][6] support the use of containers
It is worth noting that while simulation is outside of the scope of this document, most GlueX simulation jobs are run on the OSG using the same Docker container and CVMFS file system
Summary
Data transfer from JLab to NERSC is done using Globus[10][11]. JLab has a 10Gbps connection to ESnet[12], a US national research and education network provider, which limits the maximum rate of data transfer. The average rate for each of these was limited largely due to contention for the Lustre[13] file system where files are staged from tape before being transferred to NERSC. Prior to starting the first batch of the RunPeriod2018-08 campaign a dedicated data transfer node (DTN) was installed and the software configured to stage directly to it rather than Lustre. We were able to reliably fill the 10Gbps pipe with data transfers to NERSC
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
