Hierarchical Recovery of Missing Air Pollution Data via Improved Long-Short Term Context Encoder Network

Abstract

Due to equipment and transmission failures, data loss presents a key challenge to air quality monitoring. This paper attempts to recover missing air quality data from an air quality database. Leveraging adaptive updating convolutional neural networks (CNNs), we propose a novel Long-short term context encoder (ILSCE) model, which can simultaneously capture any temporal-spatial correlation and periodic variation identified from an air quality dataset. In addition, our model applies a new mechanism to automatically update both the air quality data and their corresponding masks in every single layer of CNN. Our proposed method presents three novelties. First, it hierarchically recovers any missing air quality values. Second, domain specific weekday/weekend and seasonal information are incorporated into the training model. Third, model performance is enhanced by an additional regularization term that captures the correlation between different air pollutants, thereby considering both background ambient pollution and local emissions. Our experimental study shows these three newly proposed features allow the ILSCE model to significantly outperform existing state-of-the-art imputation methods in air pollution data recovery. Furthermore, as data loss becomes more severe, with more missing data and more consecutively missing data, the superior recovery performance and greater robustness of our model become more prominent.