The XML Stream Query Processor SPEX

Abstract

Data streams are an emerging technology for data dissemination in cases where the data throughput or size makes it unfeasible to rely on the conventional approach based on storing the data before processing it. SPEX evaluates XPath queries against XML data streams. SPEX is built upon formal frameworks for (1) rewriting XPath queries into equivalent XPath queries without reverse axes and (2) correct query evaluation with polynomial combined complexity using networks of pushdown transducers. Such transducers are simple, independent, and can be connected in a flexible manner, thus allowing not only easy extensions but also extensive query optimization. Querying XML streams with SPEX consists in four steps: first, the input XPath query is rewritten into an XPath query without reverse axes. Second, the forward XPath query is compiled into a logical query plan abstracting out details of the concrete XPath syntax. Then, a physical query plan is generated by extending the logical query plan with operators for determination and collection of answers. In the last step, the XML stream is processed continuously with the physical query plan, and the output stream conveying the answers to the original query is generated progressively.