Complex Event Processor Research Articles

Design and Component Demonstration of an SFQ Complex Event Detector Corresponding to Regular Expressions

A complex event processor (CEP) is a real-time processing system for extracting meaningful data from an input data stream. One potential application of the CEP system is for a network intrusion detection system (NIDS), which monitors packet data streams in a network to detect packets containing viruses. A complex event detector (CED) circuit performs pattern matching in the CEP system, where high throughput is required. To respond to high throughput, we designed a CED circuit using single-flux-quantum (SFQ) circuits, which have the potential to operate at a very high speed beyond 50 GHz with extremely low power consumption. The proposed SFQ CED circuit corresponds to regular expressions. It consists of a string-matching unit (SMU), which performs string matching, and a state transition unit (STU), a structure mimicking a finite automaton. We designed the fundamental component of the SFQ CED circuit and demonstrated an eight-symbol CED circuit at a maximum operating frequency of 52.9 GHz.

An Analysis of Computational Resources of Event-Driven Streaming Data Flow for Internet of Things: A Case Study

Abstract Information and communication technologies backbone of a smart city is an Internet of Things (IoT) application that combines technologies such as low power IoT networks, device management, analytics or event stream processing. Hence, designing an efficient IoT architecture for real-time IoT applications brings technical challenges that include the integration of application network protocols and data processing. In this context, the system scalability of two architectures has been analysed: the first architecture, named as POST architecture, integrates the hyper text transfer protocol with an Extract-Transform-Load technique, and is used as baseline; the second architecture, named as MQTT-CEP, is based on a publish-subscribe protocol, i.e. message queue telemetry transport, and a complex event processor engine. In this analysis, SAVIA, a smart city citizen security application, has been deployed following both architectural approaches. Results show that the design of the network protocol and the data analytic layer impacts highly in the Quality of Service experimented by the final IoT users. The experiments show that the integrated MQTT-CEP architecture scales properly, keeps energy consumption limited and thereby, promotes the development of a distributed IoT architecture based on constraint resources. The drawback is an increase in latency, mainly caused by the loosely coupled communication pattern of MQTT, but within reasonable levels which stabilize with increasing workloads.

Towards a control-as-a-service architecture for smart environments

The popularization of the Internet of Things has opened opportunities for applications in various economic sectors, allowing them to monitor and control various types of environments. However, with the increasing number of data sent by devices, applications have come to demand increased availability and Internet bandwidth, but these demands are still a bottleneck in regions farther from large urban centers and with less economic development. In this work, we examine how bandwidth limitations can affect cloud-based applications and propose a Fog Computing-based Control-as-a-Service architecture to dynamically process events in the context of Internet of Things and Smart Environments. The architecture is composed of a Rules Engine and a Complex Event Processor based on Supervisory Control techniques for Discrete Event Systems. The Rules Engine allows defining dynamic rules conditioning control from different types of input data. The Complex Event Processor with automatic synthesis and reconfiguration capability allows performing control actions to established rules. Finally, we present scenarios where this solution can be applied, such as Smart Farm, Smart Home, Smart City, and Smart Healthcare.

Rules engine and complex event processor in the context of internet of things for precision agriculture

The Internet of Things (IoT) applications monitor large data flows and events in real time, some raw data is captured from devices located in wireless sensor networks (WSN) and used to make control decisions about actuators. This can be a major problem when the devices grow in number as well as the data that is captured. In this paper, we propose an architecture called “RECEP” for the dynamic processing of events generated in the context of IoT and Precision Agriculture (PA); it is made up of two components: Rules Engine (RE) and Complex Event Processor (CEP). RE allows you to configure dynamic rules conditioning input data from different sources and planning control actions on actuators, alerts, and notifications for end users or applications. The CEP component fuses the input data at the rate at which they arrive, with the rules established in the RE and it performs a prescriptive analysis that consists not only in predicting or detecting patterns of events, but in making automatic decisions. RECEP was implemented in a virtual machine with a 1.9 GHz CPU and 6 GB RAM, then it was integrated into an intelligent irrigation system of an experimental banana plot located in Machala-Ecuador. A WSN simulator was also used to generate sensor data in large quantities, the CEP was evaluated with several test cases, and results show that it consumes computational resources with a growth trend, represented by a logarithmic regression model (r-squared > 0.9); that is, the more events are processed, there is a minimum consumption of resources. It was tested for fifteen days; around 25 thousand events/s were processed. Our RECEP can be implemented in low-cost infrastructure typical of small and large banana producers.

P ea CE-Ful Web Event Extraction and Processing as Bitemporal Mutable Events

The web is the largest bulletin board of the world. Events of all types, from flight arrivals to business meetings, are announced on this board. Tracking and reacting to such event announcements, however, is a tedious manual task, only slightly alleviated by email or similar notifications. Announcements are published with human readers in mind, and updates or delayed announcements are frequent. These characteristics have hampered attempts at automatic tracking. P ea CE provides the first integrated framework for event processing on top of web event ads, consisting of event extraction, complex event processing, and action execution in response to these events. Given a schema of the events to be tracked, the framework populates this schema by extracting events from announcement sources. This extraction is performed by little programs called wrappers that produce the events including updates and retractions. P ea CE then queries these events to detect complex events, often combining announcements from multiple sources. To deal with updates and delayed announcements, P ea CE’s schemas are bitemporal, to distinguish between occurrence and detection time. This allows complex event specifications to track updates and to react upon differences in occurrence and detection time. In case of new, changing, or deleted events, P ea CE allows one to execute actions, such as tweeting or sending out email notifications. Actions are typically specified as web interactions, for example, to fill and submit a form with attributes of the triggering event. Our evaluation shows that P ea CE’s processing is dominated by the time needed for accessing the web to extract events and perform actions, allotting to 97.4%. Thus, P ea CE requires only 2.6% overhead, and therefore, the complex event processor scales well even with moderate resources. We further show that simple and reasonable restrictions on complex event specifications and the timing of constituent events suffice to guarantee that P ea CE only requires a constant buffer to process arbitrarily many event announcements.

STAR Online Framework: from Metadata Collection to Event Analysis and System Control

In preparation for the new era of RHIC running (RHIC-II upgrades and possibly, the eRHIC era), the STAR experiment is expanding its modular Message Interface and Reliable Architecture framework (MIRA). MIRA allowed STAR to integrate meta-data collection, monitoring, and online QA components in a very agile and efficient manner using a messaging infrastructure approach. In this paper, we briefly summarize our past achievements, provide an overview of the recent development activities focused on messaging patterns and describe our experience with the complex event processor (CEP) recently integrated into the MIRA framework. CEP was used in the recent RHIC Run 14, which provided practical use cases. Finally, we present our requirements and expectations for the planned expansion of our systems, which will allow our framework to acquire features typically associated with Detector Control Systems. Special attention is given to aspects related to latency, scalability and interoperability within heterogeneous set of services, various data and meta-data acquisition components coexisting in STAR online domain.

Privacy and incongruence-focused disaggregation of water consumption data in real time

While fixture-level disaggregation of water data has a number of applications in decision support systems, it also gives rise to data privacy risks. This paper presents a system of algorithms that identify and detect water activity events, and then analyse disag- gregated information to distinguish between privacy-sensitive behaviour and incongruent data due to technical faults or leakages. The process running in real-time is implemented within the Complex Event Processor (CEP) of WSO2 and the ICeWater project. Results provide insights of future directions in theoretical aspects of disaggregation, privacy and CEP system implementation.