Window Join Research Articles

PECJ: Stream Window Join on Disorder Data Streams with Proactive Error Compensation

Stream Window Join (SWJ), a vital operation in stream analytics, struggles with achieving a balance between accuracy and latency due to out-of-order data arrivals. Existing methods predominantly rely on adaptive buffering, but often fall short in performance, thereby constraining practical applications. We introduce PECJ, a solution that proactively incorporates unobserved data to enhance accuracy while reducing latency, thus requiring robust predictive modeling of stream oscillation. At the heart of PECJ lies a mathematical formulation of the posterior distribution approximation (PDA) problem using variational inference (VI). This approach circumvents error propagation while meeting the low-latency demands of SWJ. We detail the implementation of PECJ, striking a balance between complexity and generality, and discuss both analytical and learning-based approaches. Experimental evaluations reveal PECJ's superior performance. The successful integration of PECJ into a multi-threaded SWJ benchmark testbed further establishes its practical value, demonstrating promising advancements in enhancing data stream processing capabilities amidst out-of-order data.

Research on Sliding Window Join Semantics and Join Algorithm in Heterogeneous Data Streams

Research on Sliding Window Join Semantics and Join Algorithm in Heterogeneous Data Streams

An embedded co-processor for accelerating window joins over uncertain data streams

In many new applications fields, such as surveillance, networks, and sensor technologies, the generating data are time-varying, unpredictable, uncertain, continuously arriving, and must be processed online. To keep up with these rapid data streams, accelerating data processing is one of the key problems in a UDSMS (Uncertain Data Stream Management System). Most of the algorithms that are proposed in the literature are implemented in software and can potentially result in CPU saturation. To greatly promote UDSMS processing speed, we focus on accelerating join processing, which is one of the most costly operations. We propose, for the first time, a hardware co-processor called UWJSP (Uncertain data Window Join Special co-Processor) for implementation. UWJSP is composed of several processing units. Each unit executes special instructions that continuously track changing queries. We also propose an integrated method for improving the scalability and flexibility. The experimental results reveal that UWJSP can achieve an order of magnitude improvement over a software implementation. The low cost and high performance UWJSP can be used in applications where real-time performance is indispensable.

A Window Join Optimization Algorithm Based on Minimum Spanning Tree

A Window Join Optimization Algorithm Based on Minimum Spanning Tree