LambDP: Data Processing Framework for Terminal Applications in IoTs Services

Abstract

Data processing is the basis and kernel for implementing terminal applications in Internet of Things(IoTs) services. As a large amount of sensing devices need to be connected to the IoTs service, it will bring an enormous number of heterogeneous data, and the data is difficult to be directly used by terminal applications in upper layer. So we need a data processing framework to achieve data integration and processing in IoTs services. In this paper, we introduce a data processing framework, LambDP, which includes a protocol stack system, a publish/subscribe system and a data processing platform. Protocol stack system offers dynamical protocol adaptation. Publish/subscribe center is a message queue, which can provide transfer and distribution for data. Data processing platform is based on lambda theory to implement parallel computing streaming mode and batch mode of data. In addition, we use the workflow pattern to encapsulate the business logic in LambDP into components, which can provide data processing services to upper layers applications. As for the performance of the data processing framework based on Lambda, we will deploy the data processing platform to give the verification.