Random Forest Classifier for Cloud Clearing of the Operational TROPOMI XCH4 Product

Abstract

The TROPOMI XCH4 data product requires rigorous cloud filtering to achieve a product accuracy of <1%. To this end, operational XCH4 data processing has been based on SUOMI-NPP VIIRS cloud observations. However, SUOMI-NPP is nearing the end of its operational life and has encountered malfunctions in 2022 and 2023. In this study, we introduce a novel machine learning cloud-clearing approach based on a random forest classifier (RFC). The RFC is trained on collocated TROPOMI and SUOMI-NPP VIIRS data to emulate VIIRS-like cloud clearing. After training, cloud masking requires only TROPOMI data, and so becomes operationally independent of SUOMI-NPP. We demonstrate the RFC approach by applying cloud clearing to operational TROPOMI XCH4 data for August 2022, a period in which VIIRS was not operational. For validation, we analyze the TROPOMI XCH4 data at 12 TCCON stations. Comparison of cloud clearing using the RFC and the original VIIRS method reveals excellent agreement with a similar station-to-station bias (−7.4 ppb versus −5.6 ppb), a similar standard deviation of the station-to-station bias (11.6 ppb versus 12 ppb), and the same Pearson correlation coefficient of 0.9. Remarkably, the RFC cloud clearing provides a slightly higher volume of data (2182 versus 2035 daily means) and appears to have fewer outliers. Since 21 November 2023, the RFC approach is part of the operational processing chain of the European Space Agency (ESA). For now, the default practice is to utilize SNPP-VIIRS when accessible. Only in cases where VIIRS data are unavailable do we resort to the RFC cloud mask.