The Swarm Archiving Payload Data Facility, an Instance Configuration of the ESA Multi-Mission Facility

Abstract

Abstract. The Swarm mission consists of 3 satellites, each carrying an identical set of instruments. The scientific algorithms for processing are organized in 11 separate processing steps including automated product quality control. In total, the mission data consists of data products of several hundred distinct types from raw to level 2 product types and auxiliary data. The systematic production for Swarm within the ESA Archiving and Payload Data Facility (APDF) is performed up to level 2. The production up to L2 (CAT2-mature algorithm) is performed completely within the APDF. A separate systematic production chain from L1B to L2 (CAT1-evolving algorithm) is performed by an external facility (L2PS) with output files archived within the APDF as well. The APDF also performs re-processing exercises. Re-processing may start directly from the acquired data or from any other intermediate level resulting in the need for a refined product version and baseline management. Storage, dissemination and circulation functionality is configurable in the ESA generic multi-mission elements and does not require any software coding. The control of the production is more involved. While the interface towards the algorithmic entities is standardized due to the introduction of a generic IPF interface by ESA, the orchestration of the individual IPFs into the overall workflows is distinctly mission-specific and not as amenable to standardization. The ESA MMFI production management system provides extension points to integrate additional logical elements for the build-up of complex orchestrated workflows. These extension points have been used to inject the Swarm-specific production logic into the system. A noteworthy fact about the APDF is that the dissemination elements are hosted in a high bandwidth infrastructure procured as a managed service, thus affording users a considerable access bandwidth. This paper gives an overview of the Swarm APDF data flows. It describes the elements of the solution with particular focus on how the available generic multi-mission functionality of the ESA MMFI was utilized and where there was a need to implement missionspecific extensions and plug-ins. The paper concludes with some statistics on the system output during commissioning and early operational phases as well as some general considerations on the utilization of a framework like the ESA MMFI, discussing benefits and pitfalls of the approach.