Science Data Products Research Articles

From Data Processes to Data Products: Knowledge Infrastructures in Astronomy

We explore how astronomers take observational data from telescopes, process them into usable scientific data products, curate them for later use, and reuse data for further inquiry. Astronomers have invested heavily in knowledge infrastructures - robust networks of people, artifacts, and institutions that generate, share, and maintain specific knowledge about the human and natural worlds. Drawing upon a decade of interviews and ethnography, this article compares how three astronomy groups capture, process, and archive data, and for whom. The Sloan Digital Sky Survey is a mission with a dedicated telescope and instruments, while the Black Hole Group and Integrative Astronomy Group (both pseudonyms) are university-based, investigator-led collaborations. Findings are organized into four themes: how these projects develop and maintain their workflows; how they capture and archive their data; how they maintain and repair knowledge infrastructures; and how they use and reuse data products over time. We found that astronomers encode their research methods in software known as pipelines. Algorithms help to point telescopes at targets, remove artifacts, calibrate instruments, and accomplish myriad validation tasks. Observations may be reprocessed many times to become new data products that serve new scientific purposes. Knowledge production in the form of scientific publications is the primary goal of these projects. They vary in incentives and resources to sustain access to their data products. We conclude that software pipelines are essential components of astronomical knowledge infrastructures, but are fragile, difficult to maintain and repair, and often invisible. Reusing data products is fundamental to the science of astronomy, whether or not those resources are made publicly available. We make recommendations for sustaining access to data products in scientific fields such as astronomy.

Performance of the UVIT Level-2 pipeline

Performance of the Level-2 pipeline, which translates the UVIT data created by the ISRO's ground segment processing systems (Level-1) into astronomer ready scientific data products, is described. This pipeline has evolved significantly from experiences during the in orbit mission. With time, the detector modules of UVIT developed certain defects which led to occasional corruption of imaging and timing data. This article will describe the improvements and mitigation plans incorporated in the pipeline and report its efficacy and quantify the performance.

Citizen science and social movements: A case of participatory monitoring of genetically modified crops in Japan

Agrifood related social controversies tend to involve scientific issues and advocacy groups increasingly turn to citizen science (CS), participatory data taking by regular people, to produce health and environmental data. A common assumption is that CS’s value lies in the data produced, and its volume and quality decide its persuasive power. This article describes participatory monitoring of genetically engineered rapeseed (canola) in Japan to suggest that social movements can leverage CS not only for the production of scientific data but also as occasions for recruitment, political awareness-raising, and collaboration with other civic organizations. The article proposes a new framework for understanding CS–social movement relations that is multi-actor (vs. expert–laypeople dyad); process-oriented (vs. product/data-oriented); and long-term (vs. one-shot and isolated data taking). There is an increasing awareness of the diversity of CS. Even those that are led by grassroots organizations have multiple and shifting uses of data and foster varying political subjectivity among participants. The proposed framework helps to understand the dynamics that shape such heterogeneous pathways of CS.

The Ariel ground segment and instrument operations science data centre

The ground segment for the ESA M4 Ariel exoplanet space mission is introduced. The ground segment encompasses the framework necessary to support the development of the Ariel mission to launch, in-flight operations and calibration, data processing pipeline and data handling, including user support. The structure of the ground segment and assumed responsibilities between ESA and the Ariel mission consortium is explained, along with their interfaces. The operational phases for the mission are introduced, including the early commissioning/verification phases, the science operations and the calibration strategy. The smooth transition of the ground segment through the various pre/post launch mission phases to nominal operations will be paramount in guaranteeing the success, scientific return and impact of the Ariel mission. The expected science data products are defined and a representative data processing pipeline is presented.

CSES High Precision Magnetometer Data Products and Example Study of an Intense Geomagnetic Storm

AbstractProviding accurate measurement of the magnetic field intensity and its vector components is one of the primary objectives of the China Seismo‐Electromagnetic Satellite (CSES). The high precision magnetometer (HPM) payload assembled on CSES is designed to achieve this goal. In this study, the data format, naming convention, and content of the CSES HPM Level 2 scientific data products are introduced, as a reference for users who are interested in this data set. In particular, flags for potential magnetic field disturbances from the platform and payloads are discussed. Possible scientific applications are also outlined. A preliminary validation of the data is conducted through comparison with magnetic data from the ESA’s Swarm constellation, and the result demonstrates that the HPM data of CSES are of good quality. Taking the intense geomagnetic storm that occurred on August 25–26, 2018 as an example, the magnetic field variations and the expansion of the field‐aligned currents (FACs) during this storm are discussed. We finally show that the CSES HPM data can be used to derive a satellite‐derived index equivalent to the Dst index, which agrees well to the index during this event. Our analysis thus suggests a high scientific potential of the HPM data.

Introduction of the scientific application system of GECAM

The direct detection of gravitational waves has started a new era of gravitational wave astronomy. As an important method for studying the counterparts of gravitational waves, the observation and search of the multi-wavelength radiation of the electromagnetic counterpart of gravitational waves has become the research focus in the field of astronomy. With the advancement of gravitational wave detectors, the number of gravitational wave samples has dramatically increased and the detection of gravitational wave electromagnetic counterparts has become very promising. The Gravitational wave high-energy Electromagnetic Counterpart All-sky Monitor (GECAM) satellite is a full-sky monitoring telescope designed for detecting high-energy electromagnetic counterparts of gravitational wave events. The main reason for designing a scientific application system of GECAM satellites, which is proposed and designed with an emphasis on discovery, is to ensure the scientific operation of the payload of satellites and achieve the production, storage, processing, calibration, release, and analysis of scientific data. It is also designed to provide technical support and services to scientific users. Based on the exploration and research of the gravitational wave electromagnetic counterpart, this article introduces an innovative monitoring method design of the GECAM scientific application system for the gravitational electromagnetic counterpart, namely, the scheme design of the GECAM scientific application system. The design covers system architecture, interface control, main technical processes, functional composition, and software planning and design. The GECAM satellite is scheduled for launch by the end of 2020. By then, the scientific application system will be able to support the satellite’s in-orbit scientific operation and research.

Terra MODIS: 20 years of on-orbit calibration and performance

Since its launch in December 1999, Terra Moderate Resolution Imaging Spectroradiometer (MODIS) has successfully operated for more than 20 years, with its observations generating a broad range of science data products that have greatly enabled the remote sensing community and users worldwide in their studies of many key geophysical parameters of the Earth’s systems. MODIS collects data in 36 spectral bands, covering wavelengths from 0.41 to 14.4 μm, which are calibrated by a set of onboard calibrators (OBCs). Also contributing to the sensor’s mission-long on-orbit calibration and characterization are near-monthly scheduled lunar observations and multiple time series of the sensor’s responses over select ground targets at a variety of scan angles. To a large extent, the quality of MODIS data products relies strongly on the dedicated efforts to operate and calibrate the instrument, to derive and update calibration parameters, and to develop and implement new calibration strategies and algorithms in response to on-orbit changes of the sensor’s characteristics and its OBC functions. We provide an overview of the Terra MODIS on-orbit operation and calibration activities over the last 20 years, including changes made to extend and preserve the instrument and OBC functions and their operation strategies. It also illustrates the sensor’s on-orbit performance with results derived from its OBC, lunar observations, and select ground targets and discusses major changes in sensor characteristics and corrections applied to the L1B algorithms as well as calibration lookup table updates. To date, the Terra MODIS instrument and its OBCs continue to operate and function normally. Except for those identified prelaunch, most spectral bands and detectors continue to meet their specified calibration requirements. Also discussed in our paper are lessons learned from Terra MODIS operation and calibration, as well as future efforts to further extend and maintain the quality of its long-term data records.

New attenuated backscatter profile by removing the CALIOP receiver's transient response

The photomultiplier tubes (PMTs) used to measure altitude-resolved 532 nm backscatter intensity by the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) do not recover rapidly following exposure to very high light levels (e.g., from optically dense water clouds). Instead, they exhibit exponentially decaying “noise tails” that can cause substantial misestimates when determining layer base altitudes and deriving cloud extinction coefficients. Here we present a deconvolution correction algorithm to remove these noise tails in the CALIOP level 1 science data products. The corrected 532 nm attenuated backscatter profiles generated by the deconvolution algorithm are evaluated using the corresponding CALIOP observations at 1064 nm and coincident profiles measured at 532 nm by the NASA-Langley airborne high spectral resolution lidar. Results indicate that our deconvolution correction algorithm effectively removes the non-ideal PMT recovery effects on the CALIOP level 1 vertical profiles, showing excellent performance for surface and water cloud lidar returns when the attenuated backscatter coefficients are greater than 0.01 km−1 sr−1.

Phase complex of the system Na,Ca||SO4,CO3,HCO3-H2O at 100 ºC

The article discusses the results of determining the possible phase equilibria in geometric images of a five-component reciprocal water-salt system of sulfates, carbonates, sodium bicarbonates and calcium at 100 ºC with subsequent construction of its phase complex diagram. The laws that determine the structure of the phase complex diagram of this system are needed to be obtained for the production of scientific data used both as a reference material and also to create the optimal conditions for the recycling of liquid waste industrial production of aluminum-containing sulfate carbonate and bicarbonate salts of sodium and calcium. It was established that the system under study at 100ºC is characterized by the presence of 31 divariant double saturation fields, 25 monovariant trisaturation curves and 14 invariant points.

On-Orbit Calibration of Terra MODIS VIS Bands Using Polarization-Corrected Desert Observations

The Moderate Resolution Imaging Spectroradiometer (MODIS) instrument on the Terra spacecraft is completing two decades of successful Earth observations, providing the scientific community with numerous products and supporting applications, such as land surface cover, sea surface temperature, aerosol properties, and vegetation. The 20 reflective solar bands (RSBs) cover a wavelength range from 0.41 to 2.1 $\mu \text{m}$ and are calibrated primarily using a solar diffuser (SD), with lunar measurements and Earth-view (EV) response trends from desert sites used for the response versus scan-angle (RVS) characterization. Prelaunch analysis showed that a few short-wavelength RSBs of Terra MODIS are particularly sensitive to the polarization of the incident light, with on-orbit results further indicating that the polarization sensitivity has experienced changes that are wavelength, mirror side (MS), and scan-angle dependent. Although the primary calibrator, the SD, should provide an unpolarized light, supplemental inputs from the EV response trends used in the RVS characterization are impacted by the Earth scene polarization. The EV trends, uncorrected for polarization, pose a significant challenge in the RVS characterization for the short wavelength bands (3, 8, 9, and 10), therefore impacting the long-term trends and uncertainty characterization. Previous studies from the MODIS Characterization Support Team (MCST) and the National Aeronautics and Space Administration (NASA) Ocean Biology Processing Group (OBPG) have independently estimated the polarization correction to be applied on the calibrated products to mitigate the impacts to some extent. In this article, the RVS is characterized using the EV response trends after correcting for the polarization effects. Results indicate a significant improvement in the long-term trending, reduced uncertainties in the forward prediction, and a more accurate per-pixel uncertainty provided by the uncertainty index (UI) in the MODIS L1B. This enhanced approach will, therefore, be considered for implementation in the future L1B reprocessing efforts. The enhanced approach also vastly simplifies the process of adding long-term trend corrections on top of the MODIS L1B, which should streamline the science data production in the future.

Flat-fielding of Full-disk Solar Images with a Gaussian-type Diffuser

Full-disk solar chromospheric and photospheric images are widely used to monitor solar activity. Flat-fielding is a basic step during scientific data production. Here, a Gaussian-type diffuser is used to measure the flat-field of full-disk solar chromospheric and photospheric images. The effectiveness of the method is verified by theoretical simulation and practical measurement. First, the uniformity of the light source generated by the Gaussian diffuser and the sunlight, with a field of view the same as the full-disk solar image, is calculated. The full width at half maximum (FWHM) of the scatter angle of the Gaussian diffuser is one degree, such that the light source presents a Gaussian distribution. After the correction of the Gaussian background, the uniformity of the area light source is 99.6%, which can be regarded as a uniform area light source. Second, experiments are conducted on full-disk solar telescopes at Huairou solar observing station and Yangbajing station. The diffuser covers the entire entrance aperture when taking flat-field frames. Through qualitative and quantitative analysis of experimental results, it is confirmed that a revised flat-field frame can modify uniformity caused by various aspects of the telescope, including vignetting, non-uniformities of filters, and contamination of optical components.

Observation data pre-processing and scientific data products generation of POLAR

POLAR is a compact space-borne detector initially designed to measure the polarization of hard X-rays emitted from Gamma-Ray Bursts in the energy range 50–500 keV. This instrument was launched successfully onboard the Chinese space laboratory Tiangong-2 (TG-2) on 2016 September 15. After being switched on a few days later, tens of gigabytes of raw detection data were produced in-orbit by POLAR and transferred to the ground every day. Before the launch date, a full pipeline and related software were designed and developed for the purpose of quickly pre-processing all the raw data from POLAR, which include both science data and engineering data, then to generate the high level scientific data products that are suitable for later science analysis. This pipeline has been successfully applied for use by the POLAR Science Data Center in the Institute of High Energy Physics (IHEP) after POLAR was launched and switched on. A detailed introduction to the pipeline and some of the core relevant algorithms are presented in this paper.

DiSSCo as a New Regional Model for Scientific Collections in Europe

European Natural Science Collections (NSC) are part of the global natural and cultural capital and represent 80% of the world bio-and geo-diversity. Data derived from these collections underpin thousands of scholarly publications and official reports (used to support legislative and regulatory processes relating to health, food, security, sustainability and environmental change) and let to inventions and products that today play an important role in our bio-economy. In the last decades, the research practice in natural sciences changed dramatically. Advances in digital, genomic and information technologies enable natural science collections to provide new insights but also ask for changing the current operational and business models of individual collections held at local natural history museums and universities. A new business model that provides unified access to collection objects and all scientific data derived from them. Although aggregating infrastructures like the Global Biodiversity Information Facility, GenBank and Catalogue of Life now successfully aggregate data on specific data classes, the landscape remains fragmented with limited capacity to bring together this information in a systematic and robust manner and with scattered access to the physical objects. The Distributed System of Scientific Collections (DiSSCo) represents a pan-European initiative, and the largest ever agreement of natural science museums, to jointly address the fragmentation of European collections. DiSSCo is unifying European natural science collections into a coherent new research infrastructure, able to provide bio- and geo-diversity data at the scale, form and precision required by a multi-disciplinary user base in science. DiSSCo is harmonising digitisation, curation and publication processes and workflows across the scientific collections in Europe and enables linking of occurrence, genomic, chemical and morphological data classes as well as publications and experts to the physical object. In this paper we will present the socio-cultural and governance aspects of this research infrastructure. DiSSCo is receiving political support from 11 countries in Europe and will gradually change its funding model from institutional to national funding, with temporary funding from the EC to support the preparation and development. Solutions to achieve large scale digitisation are currently designed in the EC funded ICEDIG project to underpin the future large scale digitisation carried out by the countries. Unified virtual (digitisation on demand) and transnational physical access to the collections is over the next four years being developed in the EC funded SYNTHESYS+ project. The governance of DiSSCo is designed to gradually change from a steering committee composed of a few large natural history museums contributing in cash to initiate the development into a legal entity in which national consortia are represented, with a central coordination office for daily management. Each country individually decides how its entities (scientific collection facilities, research councils, governmental bodies) are organised in their national consortium. A stakeholder and user forum, Scientific Advisory Board and International Advisory Board will ensure that DiSSCo will be functional in enabling science across disciplines and within the international landscape of infrastructures. Training and short scientific missions are being developed in the MOBILISE COST Action to build capacity in FAIR data production, publication and usage of scientific collection-derived data in Europe and to initiate the socio-cultural changes needed in the collection-holding institutes. A Helpdesk is being constructed in the SYNTHESYS+ and DiSSCo Prepare projects to further facilitate the use and scientific use cases have been collected in ICEDIG to develop and facilitate e-services tailored to scientific needs.

An Overview of the Science Performances and Calibration/Validation of Joint Polar Satellite System Operational Products

The Suomi National Polar-orbiting Partnership (S-NPP) satellite, launched in October 2011, initiated a series of the next-generation weather satellites for the National Oceanic and Atmospheric Administration (NOAA) Joint Polar Satellite System (JPSS) program. The JPSS program at the Center for Satellite Applications and Research (JSTAR) leads the development of the algorithms, the calibration and validation of the products to meet the specified requirements, and long-term science performance monitoring and maintenance. All of the S-NPP products have been validated and are in successful operation. The recently launched JPSS-1 (renamed as NOAA-20) satellite is producing high-quality data products that have been available from S-NPP, along with additional products, as a direct result of the instrument upgrades and science improvements. This paper presents an overview of the JPSS product suite, the performance metrics achieved for the S-NPP, and the utilization of the products by NOAA stakeholders and user agencies worldwide. The status of NOAA-20 science data products and ongoing calibration/validation (Cal/Val) efforts are discussed for user awareness. In addition, operational implementation statuses of JPSS enterprise (multisensor and multiplatform) science algorithms for product generation and science product reprocessing efforts for the S-NPP mission are discussed.

The CYGNSS Level 1 Calibration Algorithm and Error Analysis Based on On-Orbit Measurements

The calibration algorithm used by the Cyclone Global Navigation Satellite System (CYGNSS) mission to produce version 2.1 of its Level 1 (L1) science data products is described. Changes and improvements have been made to the algorithm, relative to earlier versions, based on the first year of on-orbit result. The L1 calibration consists of two parts: first, the Level 1a (L1a) calibration converts the raw Level 0 delay Doppler maps (DDMs) of processed counts into received power in units of watts. Second, the L1a DDMs are then converted to Level 1b DDMs of bistatic radar cross section values by unwrapping the forward scattering model, which are then normalized by the surface scattering area to arrive an observation of $ \sigma _0$ . An update to the bottom up term-by-term error analysis is also presented, using on-orbit results to better quantify the accuracy of the rolled-up L1 calibration. The error analysis considers uncertainties in all known input calibration parameters. Finally, a method for calibrating the time delay of CYGNSS measurements is presented.

Development of ground pipeline system for high-level scientific data products of the Hisaki satellite mission and its application to planetary space weather

The Hisaki satellite is the first-ever space telescope mission dedicated to planetary sciences. Atmospheres and magnetospheres of our solar system planets are continuously monitored by the extreme ultraviolet (EUV) spectrometer onboard Hisaki. This paper describes a data pipeline system developed for processing high-level scientific and ancillary data products from the Hisaki mission. The telemetry data downlinked from the satellite are stored in a ground telemetry database, processed in the pipeline to imaging spectral data with a 1-min temporal resolution and ancillary data products, and then archived in a public database. The imaging spectra can be further reduced to higher-level data products for practical scientific use. For example, light curves of the power emitted from Jupiter’s aurora and plasma torus with a temporal resolution of 10-min can be reduced from the imaging spectral data; the reduced light curves reveal the transport processes of energy and mass in Jupiter’s magnetosphere and associated interplanetary solar wind conditions. Continuous monitoring with Hisaki will contribute considerably to our understanding of space weather relating to planets in our solar system.

Foreword to the Special Issue on Cyclone Global Navigation Satellite System (CYGNSS) Early on Orbit Performance

The papers in this special section examine the orbit performance pf the Cyclone Global Navigation Satellite System (CYGNSS), a constellation of eight satellites that were successfully launched on December 15, 2016 into low earth orbit. Each satellite carries a four-channel bistatic radar receiver that measures GPS signals scattered by the earth surface. Global Navigation Satellite System Reflectometry (GNSS-R) techniques are used to retrieve near surface wind speed over ocean and soil moisture over land. The measurements are unique in several respects, most notably in their ability to penetrate through high levels of precipitation, made possible by the low frequency at which GPS operates, and in the frequent revisit time and complete sampling of the diurnal cycle, made possible by the large number of satellites in a low inclination orbit. Engineering commissioning of the constellation was successfully completed in March 2017. Since then, CYGNSS science data products have been continuously produced.

40 Years of Socio-Economic Approach of Management (SEAM): What We Know and Where We Go?

Since the precursor works of Coase (1937) and then Williamson (1975) and Pfeffer (1998), it is well established that dysfunctions of management systems engender huge losses of value for companies of any size and sector that are mainly hidden by traditional information systems. However, the big issue for managers still remain the evaluation of these tremendous hidden losses and their conversion into tangible performances. The socio-economic approach of management (SEAM) and its hidden cost method were precisely created to raise such an issue. Therefore, forty years after its invention in France and its spread mainly in Europe, North and South America, it is of a great interest to study the history of SEAM and its hidden cost method through the results of its application within 1,854 companies and organizations of various sizes (staffing from 2 to 30,000 employees) in 42 countries and 72 different sectors. In particular, we explain the genesis of the conception of SEAM, linked to the fact that the classic literature in economics and management ignored a whole range of the performance of organizations called hidden costs-performances, its contributions and its limitations both practically and theoretically. In particular, we explain how and why one of the major innovation of SEAM in the field of management is to allow at the same time the improvement of tangible performances as well as the production of scientific data to develop theories. Consequently, this makes SEAM a kind of unique management consulting methodology contributing both to Evidence-Based Management and Engaged Scholarship.

ExoMars Trace Gas Orbiter (TGO) Science Ground Segment (SGS)

The ExoMars Trace Gas Orbiter (TGO) Science Ground Segment (SGS), comprised of payload Instrument Team, ESA and Russian operational centres, is responsible for planning the science operations of the TGO mission and for the generation and archiving of the scientific data products to levels meeting the scientific aims and criteria specified by the ESA Project Scientist as advised by the Science Working Team (SWT). The ExoMars SGS builds extensively upon tools and experience acquired through earlier ESA planetary missions like Mars and Venus Express, and Rosetta, but also is breaking ground in various respects toward the science operations of future missions like BepiColombo or JUICE. A productive interaction with the Russian partners in the mission facilitates broad and effective collaboration. This paper describes the global organisation and operation of the SGS, with reference to its principal systems, interfaces and operational processes.

ESA's Planetary Science Archive: Preserve and present reliable scientific data sets

The European Space Agency (ESA) Planetary Science Archive (PSA) is undergoing a significant refactoring of all its components to improve the services provided to the scientific community and the public. The PSA supports ESA's missions exploring the Solar System by archiving scientific peer-reviewed observations as well as engineering data sets. This includes the Giotto, SMART-1, Huygens, Venus Express, Mars Express, Rosetta, Exomars 2016, Exomars RSP, BepiColombo, and JUICE missions. The PSA is offering a newly designed graphical user interface which is simultaneously meant to maximize the interaction with scientific observations and also minimise the efforts needed to download these scientific observations. The PSA still offers the same services as before (i.e., FTP, documentation, helpdesk, etc.). In addition, it will support the two formats of the Planetary Data System (i.e., PDS3 and PDS4), as well as providing new ways for searching the data products with specific metadata and geometrical parameters. As well as enhanced services, the PSA will also provide new services to improve the visualisation of data products and scientific content (e.g., spectra, etc.). Together with improved access to the spacecraft engineering data sets, the PSA will provide easier access to scientific data products that will help to maximize the science return of ESA's space missions.