On construction of the air pollution monitoring service with a hybrid database converter

Abstract

Air pollution has a severe impact on human health, pollution harms human health, and people start to pay attention to how to use the monitoring system in real-time recording for analysis. To maintain smooth monitoring and analysis, we have to manage the historical data separated from the incoming data. Historical data is used when we need to analyze the data, while real-time incoming data is used to visualize the real condition. To achieve this objective, we need to collect real-time data from environmental protection open data resource. However, the data might grow faster and become huge; in this case, the relational database was not designed to process a large amount of data. Therefore, we require a database technology that can handle massive volume data, that is not only Structured Query Language (NoSQL). This method raises an important point regarding how to dump the data to NoSQL without change relational database (RDB) system. Accordingly, this paper proposed an air pollution monitoring system combines Hadoop cluster to dump data from RDB to NoSQL and data backup. By this way, it will not only reduce the performance of RDB loading but also keep the service status. Dump data to NoSQL need to process without affecting the real-time monitoring of air pollution monitoring system. In this part, we focus on without interruption web service, and it can be up to 60%, through optimizing it with dump method and backup data service, MapReduce can restart the service and distribute the database when RDB is impairing. Besides that, through three different types of conversion mode, we can get the best data conversion in our system. Finally, air pollution monitoring service provides a variant of air pollution factor as an essential basis of environment detection and analysis to serve people living in a more comfortable environment.