Architecture for Complex Event Processing Using Open Source Technologies

Abstract

Big Data, more than ever, is playing a vital role in IT decision making, with such decisions increasingly moving towards being made in real-time. Organisations are optimizing service performance, better handling capacity across overall organisations, and effectively making decisions utilising operational analytics. Realizing the full value of business data is a key challenge for today's operational analytics. Complex Event Processing (CEP) is a technique for tracking, analyzing, and processing data as an event happens and is useful for Big Data because it is intended to manage data in motion. Data in motion is processed and communicated based on business rules and processes. For decisions to be better-informed, data used for decision making has to be timely, complete, accurate, trusted, valid, reliable, and relevant. CEP utilizes data generated from moment-to-moment from different emerging sources such as sensor, sentiment, geo-locational, etc. There is a need to bridge the gap between traditional business intelligence with new Big Data technologies such as CEP. Bridging of this gap will enable organisations to become agile and data-driven so that business outcomes can be maximized by delivering better-informed decisions about a customer and delivering a better service to them. In this paper we discuss the architecture developed for CEP using open source technologies and show how CEP is applied to the use case of an Electronic Coupon Distribution Service (ECDS), using location information, past shopping/travel history, gender, likes/dislikes, etc. We further explore how different types of data such as static information (gender, age, etc.), previous history (where the person travelled to, what they bought, etc.), as well as real-time information about a customer (current location, current shopping habits, etc.) would all be utilised in CEP.