Handling Incomplete and Delayed Information Using Optimal Scheduling of Big Data Stream

Abstract

Consistently, massive volumes of sensory, transactional, and web information are persistently produced as streams, which need to be investigated online as they arrive. The arrival rate of the big data stream may vary after some time. Scheduling plays a key part in big data streaming applications in a big data stream computing environment. In this paper, optimal scheduling is proposed on big data streams to handle the incomplete and delayed information. In this process, big data stream is taken as an input. The input big data stream consists of number of data streams and each data stream consists of number of tasks. Initially, the input big data stream is analyzed and the task is selected by calculating the features such as volatility, Hurst exponent, and distance. Enthalpy value is then computed based on the extracted feature for each data streams and the computed enthalpy value is taken as a feedback ID. Finally, krill herd optimization algorithm is used for the optimal scheduling of tasks based on the generated feedback ID. The results have shown that our proposed model outperformed popular scheduling algorithms in terms of computational time, schedule time, and throughput.