Air Quality Prediction Research Articles

Air quality prediction using spatio-temporal deep learning

With the rapid urbanization, many developing countries are suffering from heavy air pollution. Predicting air quality accurately is critical for people’s outside planning and governments’ policy-making in these countries. This paper studies the air quality dataset collected in 22 stations of Beijing city, China, and reveals the spatial correlations, the temporal dependencies and feature correlations hidden in the dataset. Different from existing shallow or deep learning models that do not fully consider these correlations, we propose a spatio-temporal deep learning model called Conv1D-LSTM, which unifies 1-dimensional convolutional neural network (1D CNN) and long short-term memory (LSTM) for spatial and temporal correlation feature extraction, and uses a fully connected network (FCN) to exploit these features for air quality prediction. Moreover, this study investigates the problem of data missing in the air quality dataset and impute the missing values from spatial and temporal views. Extensive experiments are conducted by applying the proposed model to hourly PM2.5 and PM10 with single and multi-step predictions over the study area during 2014–2015. The results show the effectiveness of the proposed model and its superiority over other baseline models.

Investigation of nearby monitoring station for hourly PM2.5 forecasting using parallel multi-input 1D-CNN-biLSTM

Air quality forecasting is a hot research topic that has been widely explored by the whole society. To better understand environmental quality, numerous methods have been proposed for investigating air pollutant data. Previous studies have used deep learning-based methods for hourly PM2.5 forecasting due to its success in various pattern recognition tasks. However, only using data from the target station for PM2.5 forecasting will limit its performance. In this study, we develop an automated hourly PM2.5 forecasting model using a parallel multi-input 1D-CNN-biLSTM model by combing data from both target and nearby monitoring stations. Specifically, six single-variable biLSTM-based models are first evaluated to select the best one for the subsequent analysis. Then, a parallel multi-input 1D-CNN-biLSTM architecture is constructed, where a nearby station is selected and combined with the target station for PM2.5 forecasting. Moreover, we forecast PM2.5 with an emphasis on seasonal variability. Experimental results show that combining the nearby station with the target station can improve the forecasting performance. The best average RMSE, MAE and R2 over nine sites using our proposed system are 3.88 and 2.52, and 0.94, respectively. Furthermore, the performance for Autumn is the best in terms of RMSE and MAE with emphasis on seasonality. Therefore, the proposed parallel multi-input 1D-CNN-biLSTM model can be adopted for PM2.5 forecasting.

Using Machine Learning Methods to Forecast Air Quality: A Case Study in Macao

Despite the levels of air pollution in Macao continuing to improve over recent years, there are still days with high-pollution episodes that cause great health concerns to the local community. Therefore, it is very important to accurately forecast air quality in Macao. Machine learning methods such as random forest (RF), gradient boosting (GB), support vector regression (SVR), and multiple linear regression (MLR) were applied to predict the levels of particulate matter (PM10 and PM2.5) concentrations in Macao. The forecast models were built and trained using the meteorological and air quality data from 2013 to 2018, and the air quality data from 2019 to 2021 were used for validation. Our results show that there is no significant difference between the performance of the four methods in predicting the air quality data for 2019 (before the COVID-19 pandemic) and 2021 (the new normal period). However, RF performed significantly better than the other methods for 2020 (amid the pandemic) with a higher coefficient of determination (R2) and lower RMSE, MAE, and BIAS. The reduced performance of the statistical MLR and other ML models was presumably due to the unprecedented low levels of PM10 and PM2.5 concentrations in 2020. Therefore, this study suggests that RF is the most reliable prediction method for pollutant concentrations, especially in the event of drastic air quality changes due to unexpected circumstances, such as a lockdown caused by a widespread infectious disease.

Predictive Analysis of Air Pollution Using Machine Learning Techniques

Air pollution is a major source of worry for all living things. India has one of the world’s highest levels of air pollution. Rising population, unplanned growth, increased automotive traffic, stubble burning, industrial waste, fossil fuel combustion, powerplant emissions and a variety of other causes all contribute considerably to air pollution in developing countries. Particulate matter (PM) 2.5 is the most concerning of all air pollutants since it causes major health problems in individuals. Prediction and management of air quality have therefore become critical. Several machine learning algorithms were used in this work to examine dataset results. The results of our work suggest that for future predictions, logistic regression and autoregression can be efficaciously utilised for the analysis and forecasting of levels of PM2.5 in the future. Countries can lower the prevalence of strokes, and chronic and acute respiratory illnesses such as asthma, and lung cancer by reducing air pollution levels.

Air quality index forecast in Beijing based on CNN-LSTM multi-model

Accurate predicting the air quality trend can provide a theoretical basis for environmental protection management and decision-making. This study proposed the convolutional neural networks—long short-term memory (CNN-LSTM) model, which was proposed to improve the air quality prediction accuracy. Firstly, CNN's efficient feature extraction function was used to extract data features. Then the feature vectors were constructed into the sequence form, which was transmitted to the LSTM network. The LSTM layer learned the changing rules of air quality data to predict future data.Taking Beijing's air quality index as an example, the prediction results of the CNN-LSTM model were compared with those of auto-regressive moving average (ARMA), seasonal auto-regression integrated moving average (SARIMA), recurrent neural network (RNN), long short-term memory (LSTM) and gated recurrent unit (GRU) models. The results show that, compared with other single prediction models, the CNN-LSTM achieved the highest prediction accuracy. In particular, CNN-LSTM was compared with the SARIMA model, which is a time series representative model. The indicators of the CNN-LSTM model have been well improved. The mean absolute error (MAE) and root mean square error (RMSE) of the CNN-LSTM were reduced respectively 3.17% and 5.46%, and R2 was improved 8.45%.

Influences of stratospheric intrusions to high summer surface ozone over a heavily industrialized region in northern China

The stratospheric contribution to tropospheric ozone has long been a topic of much debate over the past few decades. In this study, we leveraged multiple datasets from surface, sounding and satellite observations to reanalysis datasets, along with a global chemical transport model (Global Nested Air Quality Prediction Modelling System, GNAQPMS) to investigate the impact of a stratospheric-to-tropospheric transport (STT) event characterized by long duration and wide range in the summer on surface high ozone episodes over heavily industrialized regions in northern China. In 14–18 August 2019, the ERA5 reanalysis datasets showed a potential vorticity (PV) tongue and a deep, upper-level trough penetrate towards 35° N over the North China Plain (NCP), indicating the occurrence of a stratospheric intrusion. From Atmospheric Infrared Sounder (AIRS) measurements, we found that the ozone-rich, stratospheric air mass had been injected into the lower altitudes. The GNAQPMS generally captured the featured layers, although there was a slight underestimation in the low troposphere. The averaged magnitudes of stratospheric contribution (O3S) and percentage (O3F) simulated by GNAQPMS were 3–20 μg m−3 and 6%–20%, respectively, while the Whole Atmosphere Community Climate Model (WACCM) indicated a higher stratospheric contribution by 3–5 μg m−3. Through this study, we give our opinions on the controversial topic of a more thorough understanding of the influence of natural processes apart from anthropogenic emissions, even in a heavily polluted region during summer.

A Deep Learning based Air Quality Prediction Technique Using Influencing Pollutants of Neighboring Locations in Smart City

The level of air pollution in smart cities plays a critical role in the community’s health and quality of life. Thus, air pollution forecasting would be beneficial and would guide citizens in avoiding exposure to dangerous emissions. The air health of a place can be diagnosed by close observation of the AQI (Air Quality Index) of that place. Moreover, the AQI of a place may have some influence on the pollutant concentration of the neighboring places. To address this issue, this work introduces a hybrid deep learning framework that is able to predict the values of a corresponding metric: AQI of smart cities. As a part of this work, two algorithms are proposed. The first one replaces the missing values in the dataset and the second one formulates the influence of the nearby places’ pollutant concentrations on the air quality of a particular place. A deep learning-based forecasting model is also proposed by combining 1D-CNN and Bi-GRU. To test the applicability of the framework, a large-scale experiment is carried out with the real-world dataset collected from New South Wales, Australia. Experimental results validate that the proposed framework provides a stable forecasting result, it confirms that the AQI of a place gets affected by the pollutant concentration of the nearby places and the comparison of forecasting result with the existing state of the art models shows that the proposed model outperforms the other models.

Editorial

It gives me great pleasure to announce the eighth regular issue of 2022. In this issue, 4 papers cover various topical aspects of computer science by 12 authors from 5 countries. In an ongoing effort to further strengthen our journal, I would like to expand the editorial board: If you are a tenured associate professor or above with a strong publication record, you are welcome to apply to join our editorial board. We are also interested in high-quality proposals for special issues on new topics and trends. As always, I would like to thank all the authors for their sound research and the editorial board for their extremely valuable review effort and suggestions for improvement. These contributions, together with the generous support of the consortium members, sustain the quality of our journal. In the second regular issue, I am very pleased to introduce the following 4 accepted articles: Sanam Fida, Nayyer Masood and Nirmal Tariq from Pakistan, and Faiza Qayyum from the Republic of Korea address in their joint research a novel hybrid ensemble clustering technique for student performance prediction. In a research collaboration between India and Greece, Banani Ghose, Zeenat Rehena and Leonidas Anthopoulos discuss a Deep Learning-based technique for predicting air quality using influencing pollutants of neighboring locations in a smart city environment. Marloes Vredenborg, Daan Sutmuller, Mariëlle den Hengst-Bruggeling, and Judith Masthoff from the Netherlands introduce an exploratory study on a system to reduce information overload and tunnel vision in homicide investigations. And last but not least, Wim Westera from the Netherlands presents three computational scoring models that take into account the number of attempts that a player makes to be successful. Enjoy Reading!

An air quality prediction model based on improved Vanilla LSTM with multichannel input and multiroute output

Long short-term memory (LSTM), especially vanilla LSTM (VLSTM), has been widely used in air quality prediction field. However, VLSTM has many more parameters, thereby making training slow and prediction performance unstable. The VLSTM network input data have not been selected for better efficiency. In this paper, we propose an air quality prediction model based on the improved VLSTM with multichannel input and multiroute output (IVLSTM-MCMR). The proposed model includes the IVLSTM and MCMR modules. The proposed IVLSTM module is developed by improving the VLSTM inner structure of VLSTM in order to reduce the number of parameters that help to accelerate the convergence. A new historical information usage approach is further proposed to obtain a stable training process. For the MCMR module, a multichannel data input model (MC) with an improved linear similarity dynamic time warping is introduced to choose the valid data as the input of IVLSTM. A multiroute output model (MR) is designed to integrate the results from MC, in which the results of different target stations with different features are output by different routes. We evaluate the proposed model with the collected data from Beijing, China, and the experimental results show that our model achieves improvements regarding the predication performance.

Prediction of air quality in Sydney, Australia as a function of forest fire load and weather using Bayesian statistics.

Smoke from Hazard Reduction Burns (HRBs) and wildfires contains pollutants that are harmful to human health. This includes particulate matter less than 2.5 μm in diameter (PM2.5), which affects human cardiovascular and respiratory systems and can lead to increased hospitalisations and premature deaths. Better models are needed to predict PM2.5 levels associated with HRBs so that agencies can properly assess smoke pollution risk and balance smoke risk with the wildfire mitigation benefits of HRBs. Given this need, our aim was to develop a probabilistic model of daily PM2.5 using Bayesian regression. We focused on the region around Sydney, Australia, which regularly has hazard reduction burning, wildfires and associated smoke. We developed two regional models (mean daily and maximum daily) from observed PM2.5, weather reanalysis and satellite fire hotspot data. The models predict that the worst PM2.5 in Sydney occurs when PM2.5 was high the previous day, there is low ventilation index (i.e. the product of wind speed and planetary boundary layer height), low temperature, west to northwest winds in the Blue Mountains, an afternoon sea breeze and large areas of HRBs are being conducted, particularly to the west and north of Sydney. A major benefit of our approach is that models are fast to run, require simple inputs and Bayesian predictions convey both predicted PM2.5 and associated prediction uncertainty. Future research could include the application of similar methods to other regions, collecting more data to improve model precision and developing Bayesian PM2.5 models for wildfires.

Evaluation of the WRF and CHIMERE models for the simulation of PM2.5 in large East African urban conurbations

Abstract. Urban conurbations of East Africa are affected by harmful levels of air pollution. The paucity of local air quality networks and the absence of the capacity to forecast air quality make difficult to quantify the real level of air pollution in this area. The CHIMERE chemistry transport model has been used along with the Weather Research and Forecasting (WRF) meteorological model to run high-spatial-resolution (2 × 2 km) simulations of hourly concentrations of particulate matter with an aerodynamic diameter smaller than 2.5 µm (PM2.5) for three East African urban conurbations: Addis Ababa in Ethiopia, Nairobi in Kenya, and Kampala in Uganda. Two existing emission inventories were combined to test the performance of CHIMERE as an air quality model for a target monthly period in 2017, and the results were compared against observed data from urban, roadside, and rural sites. The results show that the model is able to reproduce hourly and daily temporal variabilities in aerosol concentrations that are close to observed values from urban, roadside, and rural environments. CHIMERE's performance as a tool for managing air quality was also assessed. The analysis demonstrated that, despite the absence of high-resolution data and up-to-date biogenic and anthropogenic emissions, the model was able to reproduce 66 %–99 % of the daily PM2.5 exceedances above the World Health Organization (WHO) 24 h mean PM2.5 guideline (25 µg m−3) in the three cities. An analysis of the 24 h average PM2.5 levels was also carried out for 17 constituencies in the vicinity of Nairobi. This showed that 47 % of the constituencies in the area exhibited a poor Air Quality Index for PM2.5 that was in the unhealthy category for human health, thereby exposing between 10 000 and 30 000 people per square kilometre to harmful levels of air contamination.

Air quality prediction and long-term trend analysis: a case study of Beijing

Air quality prediction and long-term trend analysis: a case study of Beijing

AI‐ and IoT‐based hybrid model for air quality prediction in a smart city with network assistance

Air pollution is one of the biggest concerns in the world but it has not been paid much attention in developing countries. It is necessary to design models and methods to understand air pollution in developing countries to reduce the rate of pollution. This paper proposes an Internet of Things (IoT) and Artificial Intelligence (AI)-based hybrid model to predict the Air Quality Index (AQI) with a practical case study of the public data sets. The sensor node is deployed in the city to collect air quality data. Moreover, this sensor node connects to the cloud server for collecting data at the firebase real-time database through a WiFi/5G network embedded in the raspberry controller. Carbon monoxide (CO) and fine particular matter PM2.5 sensors are integrated within a sensor node to monitor the AQI of the regions. A Kalman fis also applied to remove unwanted noise from the data collected through the sensor node. Models namely Artificial Neural Network (ANN), Support Vector Machine (SVM), k-nearest neighbour (k-NN), Convolutional Neural Networks (CNN), Long Short Term Memory (LSTM), CNN-LSTM, ensemble model, and a proposed model, that is, CNN-LSTM-Bayesian optimization algorithm (BOA) model, have been utilised to predict the AQI. The performance evaluation of models is done through statistical parameters, such as mean absolute error (MAE), root mean square error (RMSE), coefficient of determination (R2), and accuracy score on two different public data sets and compared with the baseline models. The performance of the CNN-LSTM-BOA model is better than baseline models in terms of above-mentioned statistical parameters as the accuracy reported is more than 97 %.This study can help predict the Air Quality Index and provide sufficient time to generate warning signals in the location.

Deep learning-based gas-phase chemical kinetics kernel emulator: Application in a global air quality simulation case

The global atmospheric chemical transport model has become a key technology for air quality forecast and management. However, precise and rapid air quality simulations and forecast are frequently limited by the model’s computational performance. The gas-phase chemistry module is the most time-consuming module in air quality models because its traditional solution method is dynamically stiff. To reduce the solving time of the gas phase chemical module, we built an emulator based on a deep residual neural network emulator (NN) for Carbon Bond Mechanism Z (CBM-Z) mechanism implemented in Global Nested Air Quality Prediction Modeling System. A global high resolution cross-life multi-species dataset was built and trained to evaluate multi-species concentration changes at a single time step of CBM-Z. The results showed that the emulator could accelerate to approximately 300–750 times while maintaining an accuracy similar to that of CBM-Z module (the average correlation coefficient squared was 0.97) at the global scale. This deep learning-based emulator could adequately represent the stiff kinetics of CBM-Z, which involves 47 species and 132 reactions. The emulated ozone (O3), nitrogen oxides (NOx), and hydroxyl radical (OH) were consistent with those of the original CBM-Z module in different global regions, heights, and time. Our results suggest that data-driven emulations have great potential in the construction of hybrid models with process-based air quality models, particularly at larger scales.

Research on the Influence of Weather Patterns on Ozone Concentration: A Case Study in Tianjin

Ozone (O3) is an important secondary substance that plays a significant role in atmospheric chemistry and climate change. Although O3 is essential in the stratosphere, it is harmful to human health in the troposphere, where this study was conducted. In recent years, O3 pollution in the Beijing-Tianjin-Hebei (BTH), Yangtze River Delta (YRD), and Pearl River Delta (PRD) regions has deteriorated, which has become an important environmental problem. The generation of O3 is closely related to meteorological factors. In this study, the weather classification method was adopted to study the effect of meteorological conditions on O3 concentration. In the BTH region, Tianjin was selected as the representative city for the research. The real-time pollutants data, meteorological re-analysis data, and meteorological data in 2019 were combined for the analysis. The subjective weather classification method was adopted to investigate the effects of different weather types on O3 concentration. The backward trajectory tracking model was used to explore the characteristics and changes of O3 pollution under two extreme weather types. The results indicate there is a good correlation between O3 concentration and ambient temperature. Under the control of low pressure on the ground and the influence of southwest airflow in the upper air for Tianjin, heavy O3 pollution occurred frequently. The addition of external transport and local generation will cause high O3 values when the weather system is weak. The O3 concentration is closely related to ambient temperature. Continuous high-temperature weather is conducive to the photochemical reaction. The multi-day O3 pollution process would occur when the weather system is robust. The first and second types of extreme weather are more likely to cause persistent O3 pollution processes. Under the premise of stable emission sources, the change in weather patterns was the main reason affecting the O3 concentration. This study aims to improve O3 pollution control and air quality prediction in the BTH region and large cities in China.

Quantitative evaluation of impacts of the steadiness and duration of urban surface wind patterns on air quality

The complexity and heterogeneity of urban land surfaces result in inconsistencies in near-surface winds, which in turn influence the diffusion and dispersion of air pollutants. In this study, we classified urban surface wind fields, quantified their steadiness, duration, and influence on air quality using hourly wind observations from 50 meteorological stations, as well as hourly PM2.5 and NO2 concentrations from 18 monitoring stations during 2017–2018 in Shenzhen, a mega city in southern China. We found that the K-means clustering technique was reliable for distinguishing surface wind patterns within the city. Urban surface-wind patterns greatly affected pollutant concentrations. When dominated by calm, northerly wind, high PM2.5/NO2 concentration episodes occurred more frequently than those during other surface wind patterns. The urban surface transport index (USTI) was used to quantify the steadiness of surface wind classes. High pollutant concentrations were present during both high wind speed periods with a large USTI, indicating external pollutant transport, and during low wind speed periods with a small USTI, indicating pollutant accumulation. The threshold durations for surface wind fields (TDSWF) was proposed to quantify the impacts of surface wind persistence on air quality. We found that poor air quality occurred during the first several hours of a dominant wind pattern, indicating that transitions between wind patterns should be a particular focus when assessing air-quality deterioration. USTI and TDSWF are potentially applicable to other urban areas, owing to their clear definitions and simple calculation. In combination with wind speeds, these indices are likely to improve air quality forecasting and strategic decisions on air pollution emergencies, based on long time series of multiple wind and pollutant concentration observations.

Analysis of source distribution of high carbon monoxide events using airborne and surface observations in Korea

Carbon monoxide (CO) is a key tracer of anthropogenic pollution, and it indirectly affects climate change. Anmyeon-do (AMY, 36.54° N, 126.33° E), a World Meteorological Organization/Global Atmosphere Watch Program (WMO/GAW) regional background station in Korea, provides continuous observations of CO mole fractions at the surface and vertical profiles within 0.5–9 km that are also collected by aircraft campaigns. The vertical profiles collected in 2019 were analyzed using meteorological parameters such as vertical velocity and boundary layer height (BLH), and a strong relationship was observed among them; however, the vertical distribution, which was influenced by unexpected strong pollution plumes, did not exhibit a strong relationship with vertical velocity and BLH. On November 16, we captured a strong CO signal within the boundary layer and at approximately 2 km above the BLH, where vertical profiles of the aerosol scattering coefficient at 550 nm were simultaneously observed with the same vertical structure. To identify the origins and source types of the stratification of the pollution plume for CO and aerosol, we also analyzed surface in situ observations of particle size distributions and reactive gases (NOx, O3, and SO2) as well as the column averaged dry-air mole fraction of carbon monoxide (XCO) and measurements of aerosol optical depth obtained from the tropospheric monitoring instrument (TROPOMI) and moderate resolution imaging spectroradiometer (MODIS) satellite, respectively. The high CO event within the BLH, including observations at the surface site, was attributed to domestic burning sources. Convective uplifting of industrial fossil fuel emissions from eastern China was responsible for the high CO dry mole fraction in the low free troposphere (FT) and the elevated aerosol scattering coefficient. Statistical analyses of one-year surface in situ observations of CO reveal that CO-emission sources from eastern China contributed to extremely high CO spikes (≥80th percentile of CO or ≥ 350 ppb, approximately) in 2019. Therefore, we concluded that emissions from eastern China have a potentially leading role in the observed high pollution events at the surface and the lower FT over the AMY station in 2019. The increase in ground temperature, with the acceleration of the global warming process, can intensify the convective uplifting of pollution plumes from emission source regions to FT over East Asian regions. In addition, such an occurrence of the stratification of the atmospheric pollution plume in the troposphere should be considered in future studies on radiative forcing to improve air-quality forecasting in this region.

Z-number-based AQI in rough set theoretic framework for interpretation of air quality for different thresholds of PM2.5 and PM10.

Kolkata has a reputation for being one of the world’s most polluted cities, particularly in the post-monsoon months of October, November, and December. Diwali, a Hindu festival, coincides with these months where a large number of firecrackers are set off followed by high emissions of air pollutants. As a result, the air quality index (AQI) deteriorates to “very poor” (301 ≤ AQI ≤ 400) and “poor” (201 ≤ AQI ≤ 300) categories. This situation stays for several days to a month. The present study aims to identify the thresholds for PM2.5 and PM10 that cause the AQI of Kolkata to deteriorate to “very poor” and “poor.” For this purpose, we have used a rough set theory-based condition-decision support system to predict the aforementioned categories of AQI. We have developed a Z-number-based novel quantification measure of semantic information of AQI to assess the reliability of the outcomes, as generated from the condition-decision-based decision rules, during post-monsoon season. The result reveals the best possible forecast of AQI with linguistic summarization of the reliability or confidence for different threshold ranges of PM10 and PM2.5. Inverse-decision rules based on rough set theory are utilized to justify and validate the forecasts. The explainability of the condition-decision support system is demonstrated/visualized using a flow graph that maps rough-rule-based different decision paths between input and output with strength, certainty, and coverage. The investigation resulted in an advanced intelligent environmental decision support system (IEDSS) for air-quality prediction.

Polarization performance simulation for the GeoXO atmospheric composition instrument: NO2 retrieval impacts

Abstract. NOAA's Geostationary Extended Observations (GeoXO) constellation will continue and expand on the capabilities of the current generation of geostationary satellite systems to support US weather, ocean, atmosphere, and climate operations. It is planned to consist of a dedicated atmospheric composition instrument (ACX) to support air quality forecasting and monitoring by providing capabilities similar to missions such as TEMPO (Tropospheric Emission: Monitoring Pollution), currently planned to launch in 2023, as well as OMI (Ozone Monitoring Instrument), TROPOMI (TROPOspheric Monitoring Instrument), and GEMS (Geostationary Environment Monitoring Spectrometer) currently in operation. As the early phases of ACX development are progressing, design trade-offs are being considered to understand the relationship between instrument design choices and trace gas retrieval impacts. Some of these choices will affect the instrument polarization sensitivity (PS), which can have radiometric impacts on environmental satellite observations. We conducted a study to investigate how such radiometric impacts can affect NO2 retrievals by exploring their sensitivities to time of day, location, and scene type with an ACX instrument model that incorporates PS. The study addresses the basic steps of operational NO2 retrievals: the spectral fitting step and the conversion of slant column to vertical column via the air mass factor (AMF). The spectral fitting step was performed by generating at-sensor radiance from a clear-sky scene with a known NO2 amount, the application of an instrument model including both instrument PS and noise, and a physical retrieval. The spectral fitting step was found to mitigate the impacts of instrument PS. The AMF-related step was considered for clear-sky and partially cloudy scenes, for which instrument PS can lead to errors in interpreting the cloud content, propagating to AMF errors and finally to NO2 retrieval errors. For this step, the NO2 retrieval impacts were small but non-negligible for high NO2 amounts; we estimated that a typical high NO2 amount can cause a maximum retrieval error of 0.25×1015 molec. cm−2 for a PS of 5 %. These simulation capabilities were designed to aid in the development of a GeoXO atmospheric composition instrument that will improve our ability to monitor and understand the Earth's atmosphere.

Spatiotemporal data partitioning for distributed random forest algorithm: Air quality prediction using imbalanced big spatiotemporal data on spark distributed framework

Spatiotemporal air quality datasets are typically collected hourly in monitoring stations deployed non-uniformly across a metropolitan city. These datasets are not only big, which poses challenges on the storage and processing capacity of centralized computing systems but also imbalanced and spatially heterogeneous, which may result in biased air quality prediction. To address these challenges, we designed and developed a parallel air quality prediction system equipped with a spatiotemporal data partitioning method, a distributed machine learning algorithm, Hadoop’s distributed data storage platform and its resource scheduler/manager, and Spark’s efficient and in-memory execution environment, which is suitable for running iterative algorithms, e.g., machine learning. Our proposed spatiotemporal partitioning method accounted for imbalance and spatial heterogeneity features of big air quality data in predictive models, which comply with the load-balancing requirement of distributed computing systems. Distributed Random Forest algorithm in the H2O library of the Spark framework was selected as the distributed machine learning algorithm to develop the air quality predictive model. This algorithm is an ensemble forest with algorithm-level adjustments to perform as efficiently as possible for big imbalanced datasets. An application of the parallel quality prediction system for Tehran, Iran showed that the parallel prediction system had considerable speedup gain and improved both the overall accuracy and class precision of air quality prediction when working with imbalanced big spatiotemporal air quality datasets. A future research direction is to add data streaming and visualization functions to the system to provide rapid and reliable air quality prediction for supporting environmental health management. • A parallel air quality prediction system based on Spark and Hadoop frameworks. • A spatiotemporal big data partitioning method based on location and year of data. • Implemented H2O Distributed Random Forest Algorithm using Sparkling Water library. • Achieved overall accuracy of 68% and precision improvements for minority classes. • Observed an increasing trend in speedup gain when adding more parallel processors.