How Automation has Improved the Performance of the MMS SDC

Abstract

The Magnetospheric Multiscale (MMS) Science Data Center (SDC) at the Laboratory for Atmospheric and Space Physics (LASP), at the University of Colorado, has managed MMS science and ancillary data processing and distribution since MMS launched in March 2015. The MMS SDC employs automation in nearly every part of its operations. Automation is used to start up processing “runners” that listen on queues for new processing jobs, which are triggered by configurable timing rules, including cron and operational events, or certain data/data files being available. A separate set of SDC code then automatically creates processing jobs and tracks its progress. The MMS SDC runs processing jobs for each instrument (47 different job types in total), ranging from levels l1a to l3, for “survey” and “burst” modes, plotting, and cdf creation. The SDC runs anywhere from a few hundred to over 2,000 jobs per day (on average, 1,000 jobs per day). Processing jobs are scheduled in a few different ways, from running based on a fixe...