Stream Processing Applications Research Articles

Adaptive Scheduling of Continuous Operators for IoT Edge Analytics

In this paper, we address the problem of adaptive scheduling of data stream processing and analytics (DSPA) applications in a shared edge fog cloud (EFC) continuum with response time constraints. The focus is on handling the dynamic workload of DSPA applications caused by the variability of their input data stream rates generated by mobile IoT devices, and the dynamically available resource capacity in the EFC continuum. To address these challenges, we characterise the different types of resources in the EFC continuum, as well as the operators that make up a DSPA application. Based on this characterisation, we propose models to evaluate the response time and the cost of using the resources in the always dynamic EFC continuum. We then formulate the problem of adaptive scheduling of a DSPA application in the EFC continuum with the objective of minimising the cost of using the shared resources subject to the constraints of the response time and the available capacity of the EFC resources. We propose a heuristic algorithm that dynamically computes a new scheduling of the DSPA application, taking into account its current deployment state and the current state of the shared resources in the EFC continuum. Experimental results, using simulation, show the effectiveness of our proposed algorithm against algorithms of related work.

Boosting general-purpose stream processing with reconfigurable hardware

Reconfigurable devices such as field-programmable gate arrays (FPGAs) offer flexible solutions to workload acceleration with high energy efficiency. Despite such a potential advantage, they often reveal hard to program by application programmers. High-level synthesis languages have been developed to provide higher-level abstractions, allowing the developers to define the FPGA behavior using an imperative programming approach based on C/C++ languages. However, such approaches still leave the developer with the responsibility to harness the low-level optimizations required to develop efficient FPGA programs. Along this line, this paper introduces FSPX, a framework helping programmers to develop FPGA-accelerated data stream processing (DSP) applications. The approach provides a high-level Python API to develop the data-flow graph of operators, which is automatically translated into an efficient Vitis source code targeting Xilinx devices. The execution of the bitstreams implementing two benchmark applications showcases the efficiency of using FPGAs for DSP workloads. In general, FSPX provides, with a reasonable time-to-solution, higher performance compared with state-of-the-art DSP frameworks.

Streaming Readout and Data-Stream Processing With ERSAP

With the exponential growth in the volume and complexity of data generated at high-energy physics and nuclear physics research facilities, there is an imperative demand for innovative strategies to process this data in real or near-real-time. Given the surge in the requirement for high-performance computing, it becomes pivotal to reassess the adaptability of current data processing architectures in integrating new technologies and managing streaming data. This paper introduces the ERSAP framework, a modern solution that synergizes flow-based programming with the reactive actor model, paving the way for distributed, reactive, and high performance in data stream processing applications. Additionally, we unveil a novel algorithm focused on time-based clustering and event identification in data streams. The efficacy of this approach is further exemplified through the data-stream processing outcomes obtained from the recent beam tests of the EIC prototype calorimeter at DESY.

An adaptive load balancing strategy for stateful join operator in skewed data stream environments

As one of the most computationally intensive operations in stream processing applications, join operation can cause severe load imbalance problem when dealing with skewed data. Most of the popular solutions focused on monitoring-based dynamic balancing strategies, making it difficult to quickly adapt to the changing frequency of data stream, and sometimes failing the balancing strategies that try to address the skewed load in the cluster. To address these issues, we propose to use the prediction results of a deep reinforcement learning model and adjust the grouping strategy in advance before the frequency change of data stream. It will enable the system to quickly adapt to data stream fluctuation, while managing the resources for effective resource utilization. The following contributions are made in this paper: 1) Explore the main factors that trigger the load skewness problem in distributed stream join systems and carefully model the load balancing problem at the application level. 2) Develop a Gated Recurrent Unit Sequence to Sequence model to predict key frequency distribution of streams, and propose a dynamic grouping algorithm and a feedback-based resource elasticity scaling algorithm to solve the load imbalance problem caused by hot keys in real time. 3) Design and implement an adaptive stream join system Aj-Stream based on the prediction model and the proposed algorithm on Apache Storm. 4) Evaluate the system performance through extensive experiments on a large scale real-world dataset and multiple synthetic datasets. The experimental results demonstrate that the Aj-Stream proposed in this paper exhibits stable throughput and latency performance with both static data streams of varying skewnesses and dynamic data streams. In comparison to existing stream-connected systems, Aj-Stream demonstrated a 22.1% increase in system throughput and a 45.5% decrease in system latency when dealing with frequently fluctuating data streams.

A stream processing abstraction framework.

Real-time analysis of large multimedia streams is nowadays made efficient by the existence of several Big Data streaming platforms, like Apache Flink and Samza. However, the use of such platforms is difficult due to the fact that facilities they offer are often too raw to be effectively exploited by analysts. We describe the evolution of RAM3S, a software infrastructure for the integration of Big Data stream processing platforms, to SPAF, an abstraction framework able to provide programmers with a simple but powerful API to ease the development of stream processing applications. By using SPAF, the programmer can easily implement real-time complex analyses of massive streams on top of a distributed computing infrastructure, able to manage the volume and velocity of Big Data streams, thus effectively transforming data into value.

Hierarchical Auto-scaling Policies for Data Stream Processing on Heterogeneous Resources

Data Stream Processing (DSP) applications analyze data flows in near real-time by means of operators, which process and transform incoming data. Operators handle high data rates running parallel replicas across multiple processors and hosts. To guarantee consistent performance without wasting resources in the face of variable workloads, auto-scaling techniques have been studied to adapt operator parallelism at run-time. However, most of the effort has been spent under the assumption of homogeneous computing infrastructures, neglecting the complexity of modern environments. We consider the problem of deciding both how many operator replicas should be executed and which types of computing nodes should be acquired. We devise heterogeneity-aware policies by means of a two-layered hierarchy of controllers. While application-level components steer the adaptation process for whole applications, aiming to guarantee user-specified requirements, lower-layer components control auto-scaling of single operators. We tackle the fundamental challenge of performance and workload uncertainty, exploiting Bayesian optimization (BO) and reinforcement learning (RL) to devise policies. The evaluation shows that our approach is able to meet users’ requirements in terms of response time and adaptation overhead, while minimizing the cost due to resource usage, outperforming state-of-the-art baselines. We also demonstrate how partial model information is exploited to reduce training time for learning-based controllers.

A Grey Literature Review on Data Stream Processing applications testing

Context:The Data Stream Processing (DSP) approach focuses on real-time data processing by applying specific techniques for capturing and processing relevant data for on-the-fly results, i.e. without necessarily requiring prior storage. Like in any other software, testing plays a vital role in the quality assurance of DSP applications. However, testing such kind of software is not a simple task. In this context, some factors that make challenging testing are message temporality, parallelism, data volume, complex infrastructure, variability, and speed of messages. Objective:This work aims to map and synthesize industry knowledge and experience regarding DSP application testing. Specifically, we want to know about challenges, test purposes, test approaches, test data sources, and adopted tools. Method:To achieve the objective, we performed a Grey Literature Review (e.g., blog posts, white papers, discussion lists, lecture themes at technical events, professional social networks, software repositories, and other web-published) on testing DSP applications. We searched the grey literature using Google’s regular search engine in addition to specific searches on technical software development content websites. The selected studies were analyzed using qualitative and quantitative techniques. Results:Results are based on evidence from 154 selected sources. The challenges for testing DSP applications are the complexity of DSP applications, test infrastructure complexity, timing, and data acquisition issues. The main test objectives identified are functional suitability, performance efficiency, reliability, and maintainability. The main test approaches reported: Performance Testing, Regression Testing, Property-Based Testing, Chaos Testing, and Contract/Schema Testing. The strategies adopted by practitioners to obtain test data: Historical Data, Production Data Mirroring, Semi-Synthetic Data, and Synthetic Data. We also report 50 tools used in various testing activities, which are used for: automating infrastructure, generating test data, test utilities, dealing with timing issues, load generation, simulation, and others. Furthermore, we identified gaps and opportunities for future scientific work. Conclusion:This work selected and summarized content produced by practitioners regarding DSP application testing. We identified that knowledge, techniques, and tools intrinsic to the practice were not present in the formal literature, so this study helps reduce the gap between industry and academia on this topic. The document has delivered benefits to industry practitioners and academic researchers.

Exploiting the Common Case When Accelerating Input-Dependent Stream Processing by FPGA

FPGAs have traditionally been successful in accelerating stream processing applications where the amount and type of work performed on each record&#x2014;e.g., image, packet&#x2014;do not depend on the record&#x0027;s contents. On the other hand, accelerating &#x2018;input-dependent&#x2019; stream processing on FPGAs presents a much more challenging problem where different records in the stream can require widely different operations. It is inefficient and unnecessary to support all operations at the same throughput when the distributions of the operations are skewed. In this paper, we examine the application of the &#x201C;make the common case fast&#x201D; strategy to efficiently accelerate &#x2018;input-dependent&#x2019; stream processing on FPGAs. In particular, we study the use of <i>fast-slow path</i> and <i>early-exit</i> techniques in the design of an FPGA-accelerated network intrusion prevention system (IPS). To avoid overfitting when common-case behavior is varied, we further examine <i>compile-time re-tuning</i> and <i>runtime adaptation</i> techniques. A quantitative analysis shows that fast-slow path and early-exit techniques can save an order of magnitude of resources compared to a common-case unaware IPS design. Compile-time re-tuning to specific conditions further achieves 30&#x0025; &#x2013; 94&#x0025; BRAM savings relative to a generalized design. Adding runtime adaptation improves the zero-loss throughput by 1.43 &#x2013; 2.75 &#x00D7; compared to a fixed design.

TriJoin: A Time-Efficient and Scalable Three-Way Distributed Stream Join System

&lt;p&gt;Stream join is one of the most fundamental operations in data stream processing applications. Existing distributed stream join systems can support efficient two-way join, which is a join operation between two streams. Based the two-way join, implementing a three-way join require to be split into double two-way joins, where the second two-way join needs to wait for the join result transmitted from the first two-way join. We show through experiments that such a design raises prohibitively high processing latency. To solve this problem, we propose TriJoin, a time-efficient three-way distributed stream join system. We design a symmetric wait-free structure by symmetrically partitioning tuples and reused join. TriJoin utilizes reused join to join each new tuple with the intermediate result of the other two streams and stored tuples locally. For a new tuple, TriJoin only joins it with the intermediate result to generate the final result without waiting, greatly reducing the processing latency. In TriJoin, we design two partitioning and storage schemes according to two different forms of three-way stream join. We implement TriJoin and conduct comprehensive experiments to evaluate the performance using real-world traces. Results show that TriJoin significantly reduces the processing latency by up to 68%, compared to existing designs.&lt;/p&gt; &lt;p&gt;&amp;nbsp;&lt;/p&gt;

Latency-Aware Strategies for Deploying Data Stream Processing Applications on Large Cloud-Edge Infrastructure

Internet of Things (IoT) applications often require the processing of data streams generated by devices dispersed over a large geographical area. Traditionally, these data streams are forwarded to a distant cloud for processing, thus resulting in high application end-to-end latency. Recent work explores the combination of resources located in clouds and at the edges of the Internet, called cloud-edge infrastructure, for deploying Data Stream Processing (DSP) applications. Most previous work, however, fails to scale to very large IoT settings. This paper introduces deployment strategies for the placement of DSP applications onto cloud-edge infrastructure. The strategies split an application graph into regions and consider regions with stringent time requirements for edge placement. The proposed Aggregate End-to-End Latency Strategy with Region Patterns and Latency Awareness (AELS+RP+LA) decreases the number of evaluated resources when computing an operator's placement by considering the communication overhead across computing resources. Simulation results show that, unlike the state-of-the-art, AELS+RP+LA scales to environments with more than 100k resources with negligible impact on the application end-to-end latency.

Decentralized Control of Distributed Cloud Networks With Generalized Network Flows

Emerging distributed cloud architectures, e.g., fog and mobile edge computing, are playing an increasingly important role in the efficient delivery of real-time stream-processing applications (also referred to as augmented information services), such as industrial automation and metaverse experiences (e.g., extended reality, immersive gaming). While such applications require processed streams to be shared and simultaneously consumed by multiple users/devices, existing technologies lack efficient mechanisms to deal with their inherent multicast nature, leading to unnecessary traffic redundancy and network congestion. In this paper, we establish a unified framework for distributed cloud network control with generalized (mixed-cast) traffic flows that allows optimizing the distributed execution of the required packet processing, forwarding, and replication operations. We first characterize the enlarged multicast network stability region under the new control framework (with respect to its unicast counterpart). We then design a novel queuing system that allows scheduling data packets according to their current destination sets, and leverage Lyapunov drift-plus-penalty control theory to develop the first fully decentralized, throughput- and cost-optimal algorithm for multicast flow control. Numerical experiments validate analytical results and demonstrate the performance gain of the proposed design over existing network control policies.

Combining stream with data parallelism abstractions for multi-cores

Stream processing applications have seen an increasing demand with the raised availability of sensors, IoT devices, and user data. Modern systems can generate millions of data items per day that require to be processed timely. To deal with this demand, application programmers must consider parallelism to exploit the maximum performance of the underlying hardware resources. In this work, we introduce improvements to stream processing applications by exploiting fine-grained data parallelism (via Map and MapReduce) inside coarse-grained stream parallelism stages. The improvements are including techniques for identifying data parallelism in sequential codes, a new language, semantic analysis, and a set of definition and transformation rules to perform source-to-source parallel code generation. Moreover, we investigate the feasibility of employing higher-level programming abstractions to support the proposed optimizations. For that, we elect SPar programming model as a use case, and extend it by adding two new attributes to its language and implementing our optimizations as a new algorithm in the SPar compiler. We conduct a set of experiments in representative stream processing and data-parallel applications. The results showed that our new compiler algorithm is efficient and that performance improved by up to 108.4x in data-parallel applications. Furthermore, experiments evaluating stream processing applications towards the composition of stream and data parallelism revealed new insights. The results showed that such composition may improve latencies by up to an order of magnitude. Also, it enables programmers to exploit different degrees of stream and data parallelism to accomplish a balance between throughput and latency according to their necessity.

STRETCH: Virtual Shared-Nothing Parallelism for Scalable and Elastic Stream Processing

Stream processing applications extract value from raw data through Directed Acyclic Graphs of data analysis tasks. Shared-nothing (SN) parallelism is the de-facto standard to scale stream processing applications. Given an application, SN parallelism ins9tantiates several copies of each analysis task, making each instance responsible for a dedicated portion of the overall analysis, and relies on dedicated queues to exchange data among connected instances. On the one hand, SN parallelism can scale the execution of applications both up and out since threads can run task instances within and across processes/nodes. On the other hand, its lack of sharing can cause unnecessary overheads and hinder the scaling up when threads operate on data that could be jointly accessed in shared memory. This trade-off motivated us in studying a way for stream processing applications to leverage shared memory and boost the scale up (before the scale out) while adhering to the widely-adopted and SN-based APIs for stream processing applications. We introduce <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">STRETCH</i> , a framework that maximizes the scale up and offers instantaneous elastic reconfigurations (without state transfer) for stream processing applications. We propose the concept of Virtual Shared-Nothing (VSN) parallelism and elasticity and provide formal definitions and correctness proofs for the semantics of the analysis tasks supported by <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">STRETCH</i> , showing they extend the ones found in common Stream Processing Engines. We also provide a fully implemented prototype and show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">STRETCH</i> 's performance exceeds that of state-of-the-art frameworks such as Apache Flink and offers, to the best of our knowledge, unprecedented ultra-fast reconfigurations, taking less than 40 ms even when provisioning tens of new task instances.

A parallel programming assessment for stream processing applications on multi-core systems

Multi-core systems are any computing device nowadays and stream processing applications are becoming recurrent workloads, demanding parallelism to achieve the desired quality of service. As soon as data, tasks, or requests arrive, they must be computed, analyzed, or processed. Since building such applications is not a trivial task, the software industry must adopt parallel APIs (Application Programming Interfaces) that simplify the exploitation of parallelism in hardware for accelerating time-to-market. In the last years, research efforts in academia and industry provided a set of parallel APIs, increasing productivity to software developers. However, a few studies are seeking to prove the usability of these interfaces. In this work, we aim to present a parallel programming assessment regarding the usability of parallel API for expressing parallelism on the stream processing application domain and multi-core systems. To this end, we conducted an empirical study with beginners in parallel application development. The study covered three parallel APIs, reporting several quantitative and qualitative indicators involving developers. Our contribution also comprises a parallel programming assessment methodology, which can be replicated in future assessments. This study revealed important insights such as recurrent compile-time and programming logic errors performed by beginners in parallel programming, as well as the programming effort, challenges, and learning curve. Moreover, we collected the participants’ opinions about their experience in this study to understand deeply the results achieved.

Joint Task Offloading and Resource Allocation for IoT Edge Computing With Sequential Task Dependency

Incorporating mobile-edge computing (MEC) in the Internet of Things (IoT) enables resource-limited IoT devices to offload their computation tasks to a nearby edge server. In this article, we investigate an IoT system assisted by the MEC technique with its computation task subjected to sequential task dependency, which is critical for video stream processing and other intelligent applications. To minimize energy consumption per IoT device while limiting task processing delay, task offloading strategy, communication resource, and computation resource are optimized jointly under both slow and fast-fading channels. In slow fading channels, an optimization problem is formulated, which is nonconvex and involves one integer variable. To solve this challenging problem, we decompose it as a 1-D search of task offloading decision problem and a nonconvex optimization problem with task offloading decision given. Through mathematical manipulations, the nonconvex problem is transformed to be a convex one, which is shown to be solvable only with the simple Golden search method. In fast-fading channels, optimal online policies depending on the instant channel state are derived even though they are entangled. In addition, it is proved that the derived policy will converge to the offline policy when the channel coherence time is low, which can help save extra computation complexity. Numerical results verify the correctness of our analysis and the effectiveness of our proposed strategies over the existing methods.

Integrating Energy-Optimizing Scheduling of Moldable Streaming Tasks with Design Space Exploration for Multiple Core Types on Configurable Platforms

Design space exploration of a configurable, heterogeneous system for a given application with required throughput searches for a combination of cores or softcores with different architectures that can be accommodated within the given ASIC or FPGA area and that achieves the required throughput and optimizes power consumption. For a soft real-time streaming application, modeled as a task graph with internally parallelizable streaming tasks, this requires assigning a core type and quantity and DVFS frequency level to each task, which implies task runtime and energy consumption, and mapping and scheduling the tasks, such that the throughput requirement is met. We tightly integrate such static scheduling for stream processing applications with design space exploration of the best heterogeneous core combination, and solve the resulting combined optimization problem by an integer linear program (ILP). We evaluate our solution for different numbers of core types on synthetic and application-based task graphs, and demonstrate improvements of up to 34.8% for ARM big and LITTLE cores, and 70.5% for 3 different core types.

Revisiting the Design of Parallel Stream Joins on Trusted Execution Environments

The appealing properties of secure hardware solutions such as trusted execution environment (TEE) including low computational overhead, confidentiality guarantee, and reduced attack surface have prompted considerable interest in adopting them for secure stream processing applications. In this paper, we revisit the design of parallel stream join algorithms on multicore processors with TEEs. In particular, we conduct a series of profiling experiments to investigate the impact of alternative design choices to parallelize stream joins on TEE including: (1) execution approaches, (2) partitioning schemes, and (3) distributed scheduling strategies. From the profiling study, we observe three major high-performance impediments: (a) the computational overhead introduced with cryptographic primitives associated with page swapping operations, (b) the restrictive Enclave Page Cache (EPC) size that limits the supported amount of in-memory processing, and (c) the lack of vertical scalability to support the increasing workload often required for near real-time applications. Addressing these issues allowed us to design SecJoin, a more efficient parallel stream join algorithm that exploits modern scale-out architectures with TEEs rendering no trade-offs on security whilst optimizing performance. We present our model-driven parameterization of SecJoin and share our experimental results which have shown up to 4-folds of improvements in terms of throughput and latency.

SDN-based fog and cloud interplay for stream processing

This paper focuses on SDN-based approaches for deploying stream processing workloads on heterogeneous environments comprising wide-area networks, cloud and fog resources. Stream processing applications impose strict latency requirements to operate appropriately. Deploying workloads in the fog reduces unnecessary delays, but its computational resources may not handle all the tasks. On the other hand, offloading the tasks to the cloud is constrained by limited network resources and involves additional transmission delays that exceed latency thresholds. Adaptive workload deployment may solve these issues by ensuring that resource and latency requirements are satisfied for all the data streams processed by an application. This paper’s main contribution consists of dynamic workload placement algorithms operating on stream processing requests with latency constraints. Provisioning of computing infrastructure exploits the interplay between fog and cloud under limited network capacity. The algorithms aim to maximize the ratio of successfully handled requests by effectively utilizing available resources while meeting application latency constraints. Experiments demonstrate that the goal can be achieved by detailed analysis of requests and ensuring balanced computing and network resources utilization. As a result, up to 30% improvement over the reference algorithms in success rate is observed.

Modeling and mitigating human annotation errors to design efficient stream processing systems with human-in-the-loop machine learning

High-quality human annotations are necessary for creating effective machine learning-driven stream processing systems. We study hybrid stream processing systems based on a Human-In-The-Loop Machine Learning (HITL-ML) paradigm, in which one or many human annotators and an automatic classifier (trained at least partially by the human annotators) label an incoming stream of instances. This is typical of many near-real-time social media analytics and web applications, including annotating social media posts during emergencies by digital volunteer groups. From a practical perspective, low-quality human annotations result in wrong labels for retraining automated classifiers and indirectly contribute to the creation of inaccurate classifiers.Considering human annotation as a psychological process allows us to address these limitations. We show that human annotation quality is dependent on the ordering of instances shown to annotators and can be improved by local changes in the instance sequence/order provided to the annotators, yielding a more accurate annotation of the stream. We adapt a theoretically-motivated human error framework of mistakes and slips for the human annotation task to study the effect of ordering instances (i.e., an “annotation schedule”). Further, we propose an error-avoidance approach to the active learning paradigm for stream processing applications robust to these likely human errors (in the form of slips) when deciding a human annotation schedule. We support the human error framework using crowdsourcing experiments and evaluate the proposed algorithm against standard baselines for active learning via extensive experimentation on classification tasks of filtering relevant social media posts during natural disasters.According to these experiments, considering the order in which data instances are presented to a human annotator leads to increased accuracy for machine learning and awareness of the potential properties of human memory for the class concept, which may affect annotation for automated classifiers. Our results allow the design of hybrid stream processing systems based on the HITL-ML paradigm, which requires the same amount of human annotations, but that has fewer human annotation errors. Automated systems that help reduce human annotation errors could benefit several web stream processing applications, including social media analytics and news filtering.

SPBench: a framework for creating benchmarks of stream processing applications

SPBench: a framework for creating benchmarks of stream processing applications