Generation of Frequent sensor epochs using efficient Parallel Distributed mining algorithm in large IOT

Abstract

Numerous data mining algorithms are implemented using huge volume of sensor data to generate the frequent item sets that are useful in many aspects such as to predict the behavioural sensor patterns of future events and to detect the survival of sensors in large IOT . Traditional data mining algorithms transactional databases to produce the frequent patterns. Since the rate of input data arrival in large IOT varies, it becomes difficult to mine the dynamic sensor database. This work proposes an efficient algorithm known as Vertical Partitioning Parallel Distributed Algorithm(VPPDA) that uses MapReduce framework to mine sensor epochs to detect the survival of sensors using association rules formed by generated frequent patterns. The proposed VPPDA eliminates overhead of interprocess communication, by introducing overlapped window concept implemented in MapReduce framework combined with Vertical partitioning approach to get better progress in execution time and accurate results. Additionally pipelining processing has been implemented in MapReduce framework that still increases the performance and generates the frequent sensor epochs patterns. The concept of pipeline processing was introduced to minimize the execution time and also make the situation adaptable to any input rate of incoming sensor epochs.