Distributed Data-Processing Pipeline for Mingantu Ultrawide Spectral Radioheliograph

Abstract

The Mingantu Ultrawide Spectral Radioheliograph (MUSER) is a synthetic-aperture radio interferometer built in Ming’antu, Inner Mongolia, China. As a solar-dedicated interferometric array, the MUSER can produce high-quality radio images in a frequency range of 400 MHz–15 GHz with high temporal, spatial, and spectral resolution. Implementing of the data processing system for the MUSER is a major challenge to performing high-cadence imaging in wideband and obtaining more than two orders of higher multiple frequencies. There is an urgent need to build a pipeline for processing the massive amount of MUSER data generated each day. In this article, we present a high-performance distributed data processing pipeline (DDPP) built on the OpenCluster infrastructure for processing MUSER observational data, including data storage, preprocessing, image reconstruction, deconvolution, archiving, and real-time monitoring. We comprehensively elaborate the system architecture of the pipeline and the implementation of each subsystem. The DDPP is automatic, robust, scalable, and manageable. The processing performance under a parallel CPU/GPU hybrid system meets the requirements of MUSER data processing.