A Sensor Data Processing and Access Platform Based on Hadoop for Smart Environments

Abstract

With the rapid development of sensor technologies, in nowadays various types of sensors are deployed around us as a basis to build a smart environment. It is without doubt that how to deal with the huge amount of data collected by various sensors in an efficient way and to transform these data to useful information for people to make use of has become an important research topic. The vast amount of data produced around us such as the temperature, the road conditions and the air quality can be numerically analyzed by utilizing the cloud computing technology. In this research, we implemented a MapReduce-based sensor data processing and access platform for intelligent cities. Specifically, we focus on using the MapReduce framework to process the raw data uploaded from the sensors, and then using HBase, which is a distributed, scalable, big data store, to save the sensor data. Besides, we use the Hadoop Distributed File System (HDFS) to store the street images captured by driving recorders installed in vehicles. People can then use their Android smartphones to access the sensor data from the platform. In sum, the data processing platform we developed can be an important building block for constructing various useful applications to serve people in all sorts of smart environments.