Healthcare4VideoStorm: Making Smart Decisions Based on Storm Metrics.

Weishan Zhang,Xiufeng Chen,Qinghua Lu,Pengcheng Duan

doi:10.3390/s16040588

Weishan Zhang, Xiufeng Chen + Show 2 more

Open Access

https://doi.org/10.3390/s16040588

Copy DOI

Abstract

Storm-based stream processing is widely used for real-time large-scale distributed processing. Knowing the run-time status and ensuring performance is critical to providing expected dependability for some applications, e.g., continuous video processing for security surveillance. The existing scheduling strategies’ granularity is too coarse to have good performance, and mainly considers network resources without computing resources while scheduling. In this paper, we propose Healthcare4Storm, a framework that finds Storm insights based on Storm metrics to gain knowledge from the health status of an application, finally ending up with smart scheduling decisions. It takes into account both network and computing resources and conducts scheduling at a fine-grained level using tuples instead of topologies. The comprehensive evaluation shows that the proposed framework has good performance and can improve the dependability of the Storm-based applications.

Highlights

Large scales of data, especially streaming data, are accumulating everyday through the usage of widely-deployed video cameras, mobile devices and social networks, like Twitter and Facebook.As the biggest Big Data [1], surveillance video has become an important source for mining valuable information
We first submit the target topology and submit bottleneck-maker topologies one by one, which are categorized into two types
We do not consider the memory-intensive type of topology, because each worker starts running with a predefined memory configuration

Summary

Introduction

Especially streaming data, are accumulating everyday through the usage of widely-deployed video cameras, mobile devices and social networks, like Twitter and Facebook.As the biggest Big Data [1], surveillance video has become an important source for mining valuable information. Real-time performance is important for many smart city applications, such as automatic traffic jam detection, crime detection and criminal tracking. These kinds of applications require powerful computing resources and can adopt Storm for processing video streams. An alternative is to build a message queue as a middleware between devices and your topology We choose this method in setting up the test environment. After being generated from spouts, tuples are emitted into bolts, which can consume the input tuples, carry out processing and possibly emit new streams. Streams bridge those bolts by groupings, which specify how tuples are emitted into the downstream bolts

Methods

Results

Discussion

Conclusion