Continuous Query Language Research Articles

Distributed Chronicle for the fault diagnosis in Distributed Systems

The chronicle paradigm has been used to determine the faults in distributed systems, through the modelled of temporal relationships between observable events, which describe the patterns of the system behaviour. Normally, the diagnostic mechanisms based on chronicles are semi-centralised methods, based on the information from local diagnoses. These models have scalability problems when they are implemented in very large systems. This paper represents the system to be diagnosed as a distributed system composed of several components, and the behaviour of each component is described by its own subchronicles. A chronicle recognition module is assigned to each component, and each local diagnosis result (the recognition of a sub-chronicle) is sent (as an event) to the neighbouring components, so as it can be exploited during the recognition of other sub-chronicles. Additionally, this paper uses continuous query language (CQL), instead of the classic language to define chronicles, to give them more expressiveness.

LARS: A Logic-based framework for Analytic Reasoning over Streams

The increasing availability of streaming data has accelerated advances in information processing tools that no longer store data for static querying but push information to consumers as soon as it becomes available. Stream processing aims at providing languages and tools for data that changes at a high rate. To cope with the volume of data, query languages often extend existing approaches for static data by means of window operators that return snapshots of recent data. However, the semantics of these languages are often given only informally or operationally, which makes their analysis and comparison difficult. A formal means to express the declarative semantics of such systems seems to be missing. This lack of theory is of particular relevance for the emerging research in stream reasoning which shifts the focus from throughput to higher expressiveness. To fill this gap, we present LARS, a Logic-based framework for Analytic Reasoning over Streams. At its core, LARS formulas extend propositional logic with generic window operators and additional controls to handle temporal information. On top of this, LARS programs extend Answer Set Programming (ASP) with rich stream reasoning capabilities; the latter can be exploited to target AI applications in a streaming context, such as diagnosis, configuration or planning. Specifically, we study in this article the computational complexity of LARS formulas and programs, their relationship to Linear Temporal Logic (LTL) and the well-known Continuous Query Language (CQL). Furthermore, we discuss the modeling capabilities of LARS in notes on the SPARQL extensions C-SPARQL and CQELS, and on the interval-based approach of the complex event processing language ETALIS. We finally briefly touch available implementations, in particular, the recent prototype engines Laser and Ticker that aim for high throughput and high expressiveness, respectively. Notably, both engines specify their semantics in LARS, indicating the desired flexibility of the framework and its potential as stream reasoning language itself, which is further explored in other works.

SQLS: A Storm‐Based Query Language System for Real‐Time Stream Data Analysis

In this work, a Storm-based query language System (SQLS) is proposed for real-time data stream analysis. The system is compatible with Continuous query language (CQL) specification. It supports both continuous queries and one-time queries over streaming data, and meets the requirements of user experience (traditional SQL queries) and QoS (such as real-time and throughput). In order to better meet the requirement of throughput and enhance the processing efficiency, the load shedding algorithm and cache optimization are employed during the implementation of SQL-like operators. Finally, performance testing of the proposed SQLS has been conducted on standalone Storm platform and Storm clusters. Experimental results show that our system can not only meet the needs of users, but also extend the function of real-time streaming queries processing.

LARS: A Logic-Based Framework for Analyzing Reasoning over Streams

The recent rise of smart applications has drawn interest to logical reasoning over data streams. Different query languages and stream processing/reasoning engines were proposed. However, due to a lack of theoretical foundations, the expressivity and semantics of these diverse approaches were only informally discussed. Towards clear specifications and means for analytic study, a formal framework is needed to characterize their semantics in precise terms. We present LARS, a Logic-based framework for Analyzing Reasoning over Streams, i.e., a rule-based formalism with a novel window operator providing a flexible mechanism to represent views on streaming data. We establish complexity results for central reasoning tasks and show how the prominent Continuous Query Language (CQL) can be captured. Moreover, the relation between LARS and ETALIS, a system for complex event processing is discussed. We thus demonstrate the capability of LARS to serve as the desired formal foundation for expressing and analyzing different semantic approaches to stream processing/reasoning and engines.

Research and Application on Data Stream Management System of Expressway Real-Time Monitoring System

This paper mainly puts forward a new idea that using DSMS (data stream management system) in GIS application can improve the performance of GIS application systems. It briefly discusses the necessity of using DSMS, and introduces the related technologies about streaming data. The expressway real-time monitoring system was taken as an example of GIS application. First, it analyzes the functions realized by DSMS in detail, describing basic architecture of the system, focusing on the specific process of streaming data processing. In addition, it discusses the related technology of continuous query language and analyzes specific continuous queries of some functions. Finally, the realization of the functions shows that the expressway real-time monitoring system has a better performance and reaches real-time by using DSMS.

Content-based load shedding in multimedia data stream management system

Abstract.Overload management has become very important in public safety systems that analyse high performance multimedia data streams, especially in the case of detection of terrorist and criminal dangers. Efficient overload management improves the accuracy of automatic identification of persons suspected of terrorist or criminal activity without requiring interaction with them. We argue that in order to improve the quality of multimedia data stream processing in the public safety arena, the innovative concept of a Multimedia Data Stream Management System (MMDSMS) using load-shedding techniques should be introduced into the infrastructure to monitor and optimize the execution of multimedia data stream queries. In this paper, we present a novel content-centered load shedding framework, based on searching and matching algorithms, for analysing video tuples arriving within multimedia data streams. The framework tracks and registers all symptoms of overload, and either prevents overload before it occurs, or minimizes its effects. We have extended our Continuous Query Language (CQL) syntax to enable this load shedding technique. The effectiveness of the framework has been verified using both artificial and real data video streams collected from monitoring devices.

Monitoring high performance data streams in vertical markets: Theory and applications in public safety and healthcare

Over the last several years, monitoring high performance data stream sources has become very important in various vertical markets. For example, in the public safety sector, monitoring and automatically identifying individuals suspected of terrorist or criminal activity without physically interacting with them has become a crucial security function. In the healthcare industry, noninvasive mechanical home ventilation monitoring has allowed patients with chronic respiratory failure to be moved from the hospital to a home setting without jeopardizing quality of life. In order to improve the efficiency of large data stream processing in such applications, we contend that data stream management systems (DSMS) should be introduced into the monitoring infrastructure. We also argue that monitoring tasks should be performed by executing data stream queries defined in Continuous Query Language (CQL), which we have extended with: 1) new operators that allow creation of a sophisticated event-based alerting system through the definition of threshold schemes and threshold activity scheduling, and 2) multimedia support, which allows manipulation of continuous multimedia data streams using a similarity-based join operator which permits correlation of data arriving in multimedia streams with static content stored in a conventional multimedia database. We developed a prototype in order to assess these proposed concepts and verified the effectiveness of our framework in a lab environment.

Relational languages and data models for continuous queries on sequences and data streams

Most data stream management systems are based on extensions of the relational data model and query languages, but rigorous analyses of the problems and limitations of this approach, and how to overcome them, are still wanting. In this article, we elucidate the interaction between stream-oriented extensions of the relational model and continuous query language constructs, and show that the resulting expressive power problems are even more serious for data streams than for databases. In particular, we study the loss of expressive power caused by the loss of blocking query operators, and characterize nonblocking queries as monotonic functions on the database. Thus we introduce the notion of NB -completeness to assure that a query language is as suitable for continuous queries as it is for traditional database queries. We show that neither RA nor SQL are NB -complete on unordered sets of tuples, and the problem is even more serious when the data model is extended to support order—a sine-qua-non in data stream applications. The new limitations of SQL, compounded with well-known problems in applications such as sequence queries and data mining, motivate our proposal of extending the language with user-defined aggregates (UDAs). These can be natively coded in SQL, according to simple syntactic rules that set nonblocking aggregates apart from blocking ones. We first prove that SQL with UDAs is Turing complete. We then prove that SQL with monotonic UDAs and union operators can express all monotonic set functions computable by a Turing machine ( NB -completeness) and finally extend this result to queries on sequences ordered by their timestamps. The proposed approach supports data stream models that are more sophisticated than append-only relations, along with data mining queries, and other complex applications.

Supporting views in data stream management systems

In relational database management systems, views supplement basic query constructs to cope with the demand for “higher-level” views of data. Moreover, in traditional query optimization, answering a query using a set of existing materialized views can yield a more efficient query execution plan. Due to their effectiveness, views are attractive to data stream management systems. In order to support views over streams, a data stream management system should employ a closed (or composable) continuous query language. A closed query language is a language in which query inputs and outputs are interpreted in the same way, hence allowing query composition. This article introduces the Synchronized SQL (or SyncSQL) query language that defines a data stream as a sequence of modify operations against a relation. SyncSQL enables query composition through the unified interpretation of query inputs and outputs. An important issue in continuous queries over data streams is the frequency by which the answer gets refreshed and the conditions that trigger the refresh. Coarser periodic refresh requirements are typically expressed as sliding windows. In this article, the sliding window approach is generalized by introducing the synchronization principle that empowers SyncSQL with a formal mechanism to express queries with arbitrary refresh conditions. After introducing the semantics and syntax, we lay the algebraic foundation for SyncSQL and propose a query-matching algorithm for deciding containment of SyncSQL expressions. Then, the article introduces the Nile-SyncSQL prototype to support SyncSQL queries. Nile-SyncSQL employs a pipelined incremental evaluation paradigm in which the query pipeline consists of a set of differential operators. A cost model is developed to estimate the cost of SyncSQL query execution pipelines and to choose the best execution plan from a set of different plans for the same query. An experimental study is conducted to evaluate the performance of Nile-SyncSQL. The experimental results illustrate the effectiveness of Nile-SyncSQL and the significant performance gains when views are enabled in data stream management systems.

The CQL continuous query language: semantic foundations and query execution

CQL, a continuous query language, is supported by the STREAM prototype data stream management system (DSMS) at Stanford. CQL is an expressive SQL-based declarative language for registering continuous queries against streams and stored relations. We begin by presenting an abstract semantics that relies only on “black-box” mappings among streams and relations. From these mappings we define a precise and general interpretation for continuous queries. CQL is an instantiation of our abstract semantics using SQL to map from relations to relations, window specifications derived from SQL-99 to map from streams to relations, and three new operators to map from relations to streams. Most of the CQL language is operational in the STREAM system. We present the structure of CQL's query execution plans as well as details of the most important components: operators, interoperator queues, synopses, and sharing of components among multiple operators and queries. Examples throughout the paper are drawn from the Linear Road benchmark recently proposed for DSMSs. We also curate a public repository of data stream applications that includes a wide variety of queries expressed in CQL. The relative ease of capturing these applications in CQL is one indicator that the language contains an appropriate set of constructs for data stream processing.

Issues in data stream management

Traditional databases store sets of relatively static records with no pre-defined notion of time, unless timestamp attributes are explicitly added. While this model adequately represents commercial catalogues or repositories of personal information, many current and emerging applications require support for on-line analysis of rapidly changing data streams. Limitations of traditional DBMSs in supporting streaming applications have been recognized, prompting research to augment existing technologies and build new systems to manage streaming data. The purpose of this paper is to review recent work in data stream management systems, with an emphasis on application requirements, data models, continuous query languages, and query evaluation.