Parallelism Strategies for Big Data Delayed Transfer Entropy Evaluation

Jonas R Dourado,Jordão Natal De Oliveira Júnior,Carlos D Maciel

doi:10.3390/a12090190

Abstract

Generated and collected data have been rising with the popularization of technologies such as Internet of Things, social media, and smartphone, leading big data term creation. One class of big data hidden information is causality. Among the tools to infer causal relationships, there is Delay Transfer Entropy (DTE); however, it has a high demanding processing power. Many approaches were proposed to overcome DTE performance issues such as GPU and FPGA implementations. Our study compared different parallel strategies to calculate DTE from big data series using a heterogeneous Beowulf cluster. Task Parallelism was significantly faster in comparison to Data Parallelism. With big data trend in sight, these results may enable bigger datasets analysis or better statistical evidence.

Highlights

The amount of data being generated and collected have been rising with the popularization of technologies such as Internet of Things, social media, and smartphone [1]
The knowledge about causality in such complex systems can be useful for the evaluation of their dynamics and to their modelling since it can lead to topological simplifications
We address Delay Transfer Entropy (DTE) performance issue by using a previously not described approach to decrease DTE execution time using a Beowulf cluster; we present and execute two parallelism strategies on big data time series and compare run time difference

Summary

Introduction

The amount of data being generated and collected have been rising with the popularization of technologies such as Internet of Things, social media, and smartphone [1]. The increasing amount of data led the creation of the term big data, with one definition given by Hashem et al [2], as a set of technologies and techniques to discover hidden information from diverse, complex and massive scale datasets. One class of hidden information is causality, which Cheng et al [3] discuss and propose a framework to deal with commonly found big data biases such as confounding and sampling selection. The knowledge about causality in such complex systems can be useful for the evaluation of their dynamics and to their modelling since it can lead to topological simplifications. The assessment of interaction or coupling is subject to bias [12]

Objectives

Methods

Results

Conclusion