Data reduction based on machine learning algorithms for fog computing in IoT smart agriculture

Abstract

Smart agriculture applications that analyse and manage agricultural yield using IoT systems may suffer from intermittent operation due to cloud disconnections commonly occurring in rural areas. A fog computing solution enables the IoT system to process data faster and deal with intermittent connectivity. However, the fog needs to send a high volume of data to the cloud and this can cause link congestion with unusable data traffic. Here we propose an approach to collect and store data in a fog-based smart agriculture environment and different data reduction methods. Sixteen techniques for data reduction are investigated; eight machine learning (ML) methods combined with run-length encoding, and eight combined with Huffman encoding. Our experiment uses two real data sets, where the first contains air temperature and humidity values, and the second has soil moisture and temperature conditions. The fog filters cluster the unlabelled data using unsupervised machine learning algorithms that group data into categories according to their value ranges in all experiments. Supervised learning classification methods are also used to predict the class of data samples from these categories. After that, the fog filter compresses the identified categories using two data compression techniques, run-length encoding (RLE) and the Huffman encoding, preserving the data time series nature. Our results reveal that a k-means combined with RLE method achieved the highest reduction, where the fog needed to store and transmit only 3%–6% of the original data generated by sensors. • It is a challenge to manage a massive amount of data generated by sensors in IoT. • Combining machine learning with data compression results in a larger data reduction. • Depending on the context, the fog needs to decide which classifier should use.