Complex Event Recognition Research Articles

Complex Event Recognition meets Hierarchical Conjunctive Queries

Complex Event Recognition meets Hierarchical Conjunctive Queries

Complex Event Recognition with Symbolic Register Transducers

We present a system for Complex Event Recognition (CER) based on automata. While multiple such systems have been described in the literature, they typically suffer from a lack of clear and denotational semantics, a limitation which often leads to confusion with respect to their expressive power. In order to address this issue, our system is based on an automaton model which is a combination of symbolic and register automata. We extend previous work on these types of automata, in order to construct a formalism with clear semantics and a corresponding automaton model whose properties can be formally investigated. We call such automata Symbolic Register Transducers (SRT). The distinctive feature of SRT , compared to previous automaton models used in CER, is that they can encode patterns relating multiple input events from an event stream, without sacrificing rigor and clarity. We show how SRT can be used in CER in order to detect patterns upon streams of events, using our framework that provides declarative and compositional semantics. We also compare our SRT -based CER engine against other state-of-the-art CER systems and show that it is both more expressive and more efficient.

Complex event recognition and anomaly detection with event behavior model

Complex event recognition and anomaly detection with event behavior model

The hippocampus and implicit memory

ABSTRACT The traditional memory-systems view is that explicit (conscious) long-term memory is associated with the hippocampus and implicit (nonconscious) memory is associated with non-hippocampal brain regions. This special issue of Cognitive Neuroscience focuses on whether the hippocampus is associated with implicit memory. An empirical fMRI paper by Miller, Kennard, Gowland, Antoniades, and Rosenthal (this issue) evaluated recognition memory performance of autobiographical amnesia patients with bilateral damage to hippocampal sub-region CA3 and found they had greater than chance recognition memory performance for spatial sequence learning, spatial item learning-same location, and non-spatial item learning, but chance performance for non-spatial sequence learning and spatial item learning-different location. These results are at odds with the view that the hippocampus is generally involved in sequence learning and complex event recognition. A discussion paper by Steinkrauss and Slotnick (this issue) considered whether fMRI studies have provided evidence that the hippocampus is associated with implicit memory. It was argued that all previous studies have been confounded by explicit memory, attentional states, stimuli, or novelty. This discussion paper is followed by commentaries from Hannula (this issue), Henke and Ruch (this issue), Rosenthal (this issue), Spaak (this issue), Thakral et al. (this issue), and Züst (this issue). The articles in this special issue illustrate that the association between the hippocampus and implicit memory is under active investigation and debate. It is hoped that the evidence and discourse in this issue will provide directions for future research.

Complex Event Recognition From Discrete Sensor Data With a Discrete Event System Framework

Recognizing complex events revealed by sensor readings is an increasingly crucial task that serves as one of the foundations for system monitoring and decision-making. In this paper, we investigate the recognition problem for a class of complex events that can be represented by a sequence of discrete sensor outputs. We view the outputs of discrete sensors as sensor events. We use an automaton to describe all possible sequences of sensor events that can be generated in a given system. The complex events to be recognized is a set of sequences of sensor events and can be represented by the marked language of another automaton. For a given sensor event sequence, we introduce the notion of “partition” to stand for a possible complex event sequence. By constructing an augmented automaton that includes all possible partitions, we find a necessary and sufficient condition for the recognition problem to be solvable. We then find an algorithm to check the condition. Finally, an online complex event recognition procedure is proposed to determine the occurred complex events when the complex event recognition problem is solvable.

CORE

Complex Event Recognition (CER) systems are a prominent technology for finding user-defined query patterns over large data streams in real time. CER query evaluation is known to be computationally challenging, since it requires maintaining a set of partial matches, and this set quickly grows super-linearly in the number of processed events. We present CORE, a novel COmplex event Recognition Engine that focuses on the efficient evaluation of a large class of complex event queries, including time windows as well as the partition-by event correlation operator. This engine uses a novel automaton-based evaluation algorithm that circumvents the super-linear partial match problem: under data complexity, it takes constant time per input event to maintain a data structure that compactly represents the set of partial matches and, once a match is found, the query results may be enumerated from the data structure with output-linear delay. We experimentally compare CORE against state-of-the-art CER systems on real-world data. We show that (1) CORE's performance is stable with respect to both query and time window size, and (2) CORE outperforms the other systems by up to five orders of magnitude on different workloads.

A Formal Framework for Complex Event Recognition

Complex event recognition (CER) has emerged as the unifying field for technologies that require processing and correlating distributed data sources in real time. CER finds applications in diverse domains, which has resulted in a large number of proposals for expressing and processing complex events. Existing CER languages lack a clear semantics, however, which makes them hard to understand and generalize. Moreover, there are no general techniques for evaluating CER query languages with clear performance guarantees.In this article, we embark on the task of giving a rigorous and efficient framework to CER. We propose a formal language for specifying complex events, calledcomplex event logic(CEL), that contains the main features used in the literature and has a denotational and compositional semantics. We also formalize the so-called selection strategies, which had only been presented as by-design extensions to existing frameworks. We give insight into the language design trade-offs regarding the strict sequencing operators of CEL and selection strategies.With a well-defined semantics at hand, we discuss how to efficiently process complex events by evaluating CEL formulas with unary filters. We start by introducing a formal computational model for CER, calledcomplex event automata(CEA), and study how to compile CEL formulas with unary filters into CEA. Furthermore, we provide efficient algorithms for evaluating CEA over event streams using constant time per event followed by output-linear delay enumeration of the results.

Complex event forecasting with prediction suffix trees

Complex event recognition (CER) systems have become popular in the past two decades due to their ability to “instantly” detect patterns on real-time streams of events. However, there is a lack of methods for forecasting when a pattern might occur before such an occurrence is actually detected by a CER engine. We present a formal framework that attempts to address the issue of complex event forecasting (CEF). Our framework combines two formalisms: (a) symbolic automata which are used to encode complex event patterns and (b) prediction suffix trees which can provide a succinct probabilistic description of an automaton’s behavior. We compare our proposed approach against state-of-the-art methods and show its advantage in terms of accuracy and efficiency. In particular, prediction suffix trees, being variable-order Markov models, have the ability to capture long-term dependencies in a stream by remembering only those past sequences that are informative enough. We also discuss how CEF solutions should be best evaluated on the quality of their forecasts.

Video Imprint.

A new unified video analytics framework (ER3) is proposed for complex event retrieval, recognition and recounting, based on the proposed video imprint representation, which exploits temporal correlations among image features across video frames. With the video imprint representation, it is convenient to reverse map back to both temporal and spatial locations in video frames, allowing for both key frame identification and key areas localization within each frame. In the proposed framework, a dedicated feature alignment module is incorporated for redundancy removal across frames to produce the tensor representation, i.e., the video imprint. Subsequently, the video imprint is individually fed into both a reasoning network and a feature aggregation module, for event recognition/recounting and event retrieval tasks, respectively. Thanks to its attention mechanism inspired by the memory networks used in language modeling, the proposed reasoning network is capable of simultaneous event category recognition and localization of the key pieces of evidence for event recounting. In addition, the latent structure in our reasoning network highlights the areas of the video imprint, which can be directly used for event recounting. With the event retrieval task, the compact video representation aggregated from the video imprint contributes to better retrieval results than existing state-of-the-art methods.

Efficient Query Processing for Dynamically Changing Datasets

The ability to efficiently analyze changing data is a key requirement of many real-time analytics applications. Traditional approaches to this problem were developed around the notion of Incremental View Maintenance (IVM), and are based either on the materialization of subresults (to avoid their recomputation) or on the recomputation of subresults (to avoid the space overhead of materialization). Both techniques are suboptimal: instead of materializing results and subresults, one may also maintain a data structure that supports efficient maintenance under updates and from which the full query result can quickly be enumerated. In two previous articles, we have presented algorithms for dynamically evaluating queries that are easy to implement, efficient, and can be naturally extended to evaluate queries from a wide range of application domains. In this paper, we discuss our algorithm and its complexity, explaining the main components behind its efficiency. Finally, we show experiments that compare our algorithm to a state-of-the-art (Higher-order) IVM engine, as well as to a prominent complex event recognition engine. Our approach outperforms the competitor systems by up to two orders of magnitude in processing time, and one order in memory consumption.

Complex event recognition in the Big Data era: a survey

The concept of event processing is established as a generic computational paradigm in various application fields. Events report on state changes of a system and its environment. Complex event recognition (CER) refers to the identification of composite events of interest, which are collections of simple, derived events that satisfy some pattern, thereby providing the opportunity for reactive and proactive measures. Examples include the recognition of anomalies in maritime surveillance, electronic fraud, cardiac arrhythmias and epidemic spread. This survey elaborates on the whole pipeline from the time CER queries are expressed in the most prominent languages, to algorithmic toolkits for scaling-out CER to clustered and geo-distributed architectural settings. We also highlight future research directions.

Математичні методи розпізнавання надзвичайних ситуацій в умовах невизначеності

Показано, що системи розпізнавання надзвичайних подій виявляють різні типи невизначеності: неповні потоки даних, помилки в потоках даних і невідповідні шаблони складних подій. Показано, що потоки подій, що потрапляють на вхід системи розпізнавання складних подій, характеризуються певним ступенем невизначеності. Джерела даних є неоднорідними і характеризуються різною структуризацією даних і відповідними процедурами реагування на пошкоджені блоки даних. Навіть для даних, визначених достатньо точно, система може некоректно моделювати складні події, що призводить до подальшого типу невизначеності. Отже, зазначено, що важливо розглянути методи розпізнавання складних подій, які можна віднести до невизначених. З цією метою було запропоновано відповідні модельні об'єкти. Проведений аналіз ключових моментів побудови систем розпізнавання складних подій, які здатні ефективно працювати в умовах невизначеності, охоплював методи стохастичного моделювання, моделі часового представлення та реляційні моделі. Розглянуто методики, що базуються на абстрактних автоматах, імовірнісних моделях графів, системах логіки першого порядку, мережах Петрі та прихованих мережах Петрі. Зазначено, що проміжним етапом роботи відповідних алгоритмів має бути створення ієрархії складних об'єктів, що не завжди піддаються чіткому визначенню. Виявлено низку обмежень щодо використовуваного синтаксису, моделей і ефективності, які були зіставлені з конкретними варіантами їх реалізації. Запропоновано підхід щодо переходу від детерміністичного математичного апарату до системи розпізнавання складних подій в умовах невизначеності, через введення функції вірогідності події. Розроблена методологія дала змогу виділити напрями досліджень і оцінити продуктивність використовуваних математичних методів.

The Complex Event Recognition Group

The Complex Event Recognition (CER) group is a research team, affiliated with the National Centre of Scientific Research "Demokritos" in Greece. The CER group works towards advanced and efficient methods for the recognition of complex events in a multitude of large, heterogeneous and interdependent data streams. Its research covers multiple aspects of complex event recognition, from efficient detection of patterns on event streams to handling uncertainty and noise in streams, and machine learning techniques for inferring interesting patterns. Lately, it has expanded to methods for forecasting the occurrence of events. It was founded in 2009 and currently hosts eight senior and junior researchers, working regularly with under-graduate students.

Parallel online event calculus learning for complex event recognition

Logic-based event recognition systems infer occurrences of events in time using a set of event definitions in the form of first-order rules. The Event Calculus is a temporal logic that has been used as a basis in event recognition applications, providing, among others, direct connections to machine learning, via Inductive Logic Programming (ILP). OLED is a recently proposed ILP system that learns event definitions in the form of Event Calculus theories, in a single pass over a data stream. We present two strategies for parallel online learning with OLED. We evaluate our proposed approaches on three datasets from the domains of activity recognition and maritime surveillance and show that they can significantly reduce training times, while they are capable of achieving super-linear speed-ups under certain circumstances.

Probabilistic Complex Event Recognition

Complex event recognition (CER) applications exhibit various types of uncertainty, ranging from incomplete and erroneous data streams to imperfect complex event patterns. We review CER techniques that handle, to some extent, uncertainty. We examine techniques based on automata, probabilistic graphical models, and first-order logic, which are the most common ones, and approaches based on Petri nets and grammars, which are less frequently used. Several limitations are identified with respect to the employed languages, their probabilistic models, and their performance, as compared to the purely deterministic cases. Based on those limitations, we highlight promising directions for future work.

Complex event recognition in the big data era

The concept of event processing is established as a generic computational paradigm in various application fields, ranging from data processing in Web environments, over maritime and transport, to finance and medicine. Events report on state changes of a system and its environment. Complex Event Recognition (CER) in turn, refers to the identification of complex/composite events of interest, which are collections of simple events that satisfy some pattern, thereby providing the opportunity for reactive and proactive measures. Examples include the recognition of attacks in computer network nodes, human activities on video content, emerging stories and trends on the Social Web, traffic and transport incidents in smart cities, fraud in electronic marketplaces, cardiac arrhythmias, and epidemic spread. In each scenario, CER allows to make sense of Big event Data streams and react accordingly. The goal of this tutorial is to provide a step-by-step guide for realizing CER in the Big Data era. To do so, it elaborates on major challenges and describes algorithmic toolkits for optimized manipulation of event streams characterized by high volume, velocity and/or lack of veracity, placing emphasis on distributed CER over potentially heterogeneous (data variety) event sources. Finally, we highlight future research directions in the field.

Detection of complex video events through visual rhythm

The recognition of complex events in videos has currently several important applications, particularly due to the wide availability of digital cameras in environments such as airports, train and bus stations, shopping centers, stadiums, hospitals, schools, buildings, roads, among others. Advances in digital technology have enhanced the capabilities for detection of video events through the development of devices with high resolution, small physical size, and high sampling rates. This work presents and evaluates the use of feature descriptors extracted from visual rhythms of video sequences in three computer vision problems: abnormal event detection, human action classification, and gesture recognition. Experiments conducted on well-known public datasets demonstrate that the method produces promising results.

Web Video Event Recognition by Semantic Analysis From Ubiquitous Documents.

In recent years, the task of event recognition from videos has attracted increasing interest in multimedia area. While most of the existing research was mainly focused on exploring visual cues to handle relatively small-granular events, it is difficult to directly analyze video content without any prior knowledge. Therefore, synthesizing both the visual and semantic analysis is a natural way for video event understanding. In this paper, we study the problem of Web video event recognition, where Web videos often describe large-granular events and carry limited textual information. Key challenges include how to accurately represent event semantics from incomplete textual information and how to effectively explore the correlation between visual and textual cues for video event understanding. We propose a novel framework to perform complex event recognition from Web videos. In order to compensate the insufficient expressive power of visual cues, we construct an event knowledge base by deeply mining semantic information from ubiquitous Web documents. This event knowledge base is capable of describing each event with comprehensive semantics. By utilizing this base, the textual cues for a video can be significantly enriched. Furthermore, we introduce a two-view adaptive regression model, which explores the intrinsic correlation between the visual and textual cues of the videos to learn reliable classifiers. Extensive experiments on two real-world video data sets show the effectiveness of our proposed framework and prove that the event knowledge base indeed helps improve the performance of Web video event recognition.

Online event recognition from moving vessel trajectories

We present a system for online monitoring of maritime activity over streaming positions from numerous vessels sailing at sea. It employs an online tracking module for detecting important changes in the evolving trajectory of each vessel across time, and thus can incrementally retain concise, yet reliable summaries of its recent movement. In addition, thanks to its complex event recognition module, this system can also offer instant notification to marine authorities regarding emergency situations, such as risk of collisions, suspicious moves in protected zones, or package picking at open sea. Not only did our extensive tests validate the performance, efficiency, and robustness of the system against scalable volumes of real-world and synthetically enlarged datasets, but its deployment against online feeds from vessels has also confirmed its capabilities for effective, real-time maritime surveillance.

Deep Fusion of Multiple Semantic Cues for Complex Event Recognition.

We present a deep learning strategy to fuse multiple semantic cues for complex event recognition. In particular, we tackle the recognition task by answering how to jointly analyze human actions (who is doing what), objects (what), and scenes (where). First, each type of semantic features (e.g., human action trajectories) is fed into a corresponding multi-layer feature abstraction pathway, followed by a fusion layer connecting all the different pathways. Second, the correlations of how the semantic cues interacting with each other are learned in an unsupervised cross-modality autoencoder fashion. Finally, by fine-tuning a large-margin objective deployed on this deep architecture, we are able to answer the question on how the semantic cues of who, what, and where compose a complex event. As compared with the traditional feature fusion methods (e.g., various early or late strategies), our method jointly learns the essential higher level features that are most effective for fusion and recognition. We perform extensive experiments on two real-world complex event video benchmarks, MED'11 and CCV, and demonstrate that our method outperforms the best published results by 21% and 11%, respectively, on an event recognition task.