Modern Data-intensive Applications Research Articles

Practical Reasoning in DatalogMTL

Abstract DatalogMTL is an extension of Datalog with metric temporal operators that has found an increasing number of applications in recent years. Reasoning in DatalogMTL is, however, of high computational complexity, which makes reasoning in modern data-intensive applications challenging. In this paper we present a practical reasoning algorithm for the full DatalogMTL language, which we have implemented in a system called MeTeoR. Our approach effectively combines an optimised (but generally non-terminating) materialisation (a.k.a. forward chaining) procedure, which provides scalable behaviour, with an automata-based component that guarantees termination and completeness. To ensure favourable scalability of the materialisation component, we propose a novel seminaïve materialisation procedure for DatalogMTL enjoying the non-repetition property, which ensures that each rule instance will be applied at most once throughout its entire execution. Moreover, our materialisation procedure is enhanced with additional optimisations which further reduce the number of redundant computations performed during materialisation by disregarding rules as soon as it is certain that they cannot derive new facts in subsequent materialisation steps. Our extensive evaluation supports the practicality of our approach.

Enhancing Hadoop distributed storage efficiency using multi-agent systems

<p>Distributed storage systems play a pivotal role in modern data-intensive applications, with Hadoop distributed file system (HDFS) being a prominent example. However, optimizing the efficiency of such systems remains a complex challenge. This research paper presents a novel approach to enhance the efficiency of distributed storage by leveraging multi-agent systems (MAS). Our research is centered on enhancing the efficiency of the HDFS by incorporating intelligent agents that can dynamically assign storage tasks to nodes based on their performance characteristics. Utilizing a decentralized decision-making framework, the suggested approach based on MAS considers the real-time performance of nodes and allocates storage tasks adaptively. This strategy aims to alleviate performance bottlenecks and minimize data transfer latency. Through extensive experimental evaluation, we demonstrate the effectiveness of our approach in improving HDFS performance in terms of data storage, retrieval, and overall system efficiency. The results reveal significant reductions in job execution times and enhanced resource utilization, there by offering a promising avenue for enhancing the efficiency of distributed storage systems.</p>

An empirical study of vulnerabilities in edge frameworks to support security testing improvement

Edge computing is a distributed computing paradigm aiming at ensuring low latency in modern data intensive applications (e.g., video streaming and IoT). It consists of deploying computation and storage nodes close to the end-users. Unfortunately, being distributed and close to end-users, Edge systems have a wider attack surface (e.g., they may be physically reachable) and are more complex to update than other types of systems (e.g., Cloud systems) thus requiring thorough security testing activities, possibly tailored to be cost-effective. To support the development of effective and automated Edge security testing solutions, we conducted an empirical study of vulnerabilities affecting Edge frameworks. The study is driven by eight research questions that aim to determine what test triggers, test harnesses, test oracles, and input types should be considered when defining new security testing approaches dedicated to Edge systems. preconditions and inputs leading to a successful exploit, the security properties being violated, the most frequent vulnerability types, the software behaviours and developer mistakes associated to these vulnerabilities, and the severity of Edge vulnerabilities. We have inspected 147 vulnerabilities of four popular Edge frameworks. Our findings indicate that vulnerabilities slip through the testing process because of the complexity of the Edge features. Indeed, they can’t be exhaustively tested in-house because of the large number of combinations of inputs, outputs, and interfaces to be tested. Since we observed that most of the vulnerabilities do not affect the system integrity and, further, only one action (e.g., requesting a URL) is sufficient to exploit a vulnerability

Eukv: Enabling Efficient Updates for Hybrid PM-DRAM Key-Value Store

Key-value store, a fundamental component in a variety of modern data-intensive applications, can benefit from the emerging persistent memory (PM) to achieve high performance. Recent research proposes hybrid PM-DRAM key-value stores, which persist key-value pairs to PM in a log-structured approach for efficient storage and maintain a volatile hash index in DRAM for fast indexing. However, they fail to achieve high performance under update-intensive workloads due to the unawareness of PM’s characteristics. We propose Eukv, a hybrid PM-DRAM key-value store that supports efficient updates. Eukv (1) leverages out-of-place updates and lazy deletions to reduce small random PM reads and writes, (2) performs concurrency control only in a volatile hash index to further reduce small random PM reads and writes introduced by persistent lock operations, (3) uses cacheline-aligned writes to take full advantage of PM’s write performance, (4) improves garbage collection efficiency through hot and cold data separation, (5) employs multi-threaded garbage collection technology to reduce the impact of inefficient garbage collection on updates at high-capacity utilization. We conduct extensive evaluations on an 18-core machine equipped with Intel Optane DC Persistent Memory Modules. Experimental results demonstrate that Eukv obtains up to 5.6× speedup than the state-of-the-art hybrid PM-DRAM key-value store.

Process variation aware DRAM-Cache resizing

As the demand for larger-sized Last Level Cache (LLC) grows due to modern data-intensive applications, employing low-density SRAM technology to create the LLC for the multicore system is no longer advantageous. Because capacity is more essential than latency in LLC, employing high-density DRAM technology to design LLC is an alternative approach. However, in order to employ DRAM as an LLC, several of its fundamental disadvantages must be effectively addressed. The DRAM-LLC is typically used in die-stacking technology, where the LLC is layered on top of the core layer. The LLC is also separated into several banks. Though larger-sized LLCs are preferred for data-intensive applications, they are not required for all applications. To lower the energy consumption of DRAM-LLC, the size of the LLC can be reduced by shutting down some unused component (bank) of the LLC. These powered-off banks can be reopened whenever the LLC capacity is increased. LLC resizing is the name of the procedure. Because of manufacturing variance, the entire DRAM-LLC does not react evenly. The section affected by process variation has higher delay and energy consumption than the other portions. In this paper, we propose a Process Variation Aware LLC Resizing (PVAR), in which the shrinkage is accomplished by shutting down the affected banks. The existing resizing strategies primarily target underused banks, but these underused banks may be healthy, and instead of powering down these banks, powering down the affected bank is more useful. Experiment results demonstrate that the suggested LLC resizing reduces energy usage by up to 47% more than existing solutions.

Unleashing the power of querying streaming data in a temporal database world: A relational algebra approach

Modern data-intensive applications have to manage huge quantities of streaming/relational data and need advanced query capabilities involving combinations of continuous queries (CQs) and one-time queries (OTQs) also requiring the verification of complex temporal conditions.In this paper, we go beyond the disjointed panorama of current approaches and adopt a new holistic approach to the integration of stream processing capabilities into the temporal database world based on the streaming table concept. To this end, we propose a full-fledged query interface composed of a TSQL2-like query language with an underlying algebraic framework. The algebraic framework, which is aimed at implementing the query interface on top of a working DBMS, is made up of: (a) the extended temporal algebra TA⋆ supporting OTQs with an hybrid temporal semantics (sequenced and non-sequenced); (b) the continuous temporal algebra CTA that extends TA⋆ with window expressions for CQ specification; (c) the translation of CTA expressions into TA⋆ ones that can be executed by a traditional DBMS with an extended kernel.

FPGA-Based Near-Memory Acceleration of Modern Data-Intensive Applications

Modern data-intensive applications demand high computation capabilities with strict power constraints. Unfortunately, such applications suffer from a significant waste of both execution cycles and energy in current computing systems due to the costly data movement between the computation units and the memory units. Genome analysis and weather prediction are two examples of such applications. Recent FPGAs couple a reconfigurable fabric with high-bandwidth memory (HBM) to enable more efficient data movement and improve overall performance and energy efficiency. This trend is an example of a paradigm shift to near-memory computing. We leverage such an FPGA with high-bandwidth memory (HBM) for improving the pre-alignment filtering step of genome analysis and representative kernels from a weather prediction model. Our evaluation demonstrates large speedups and energy savings over a high-end IBM POWER9 system and a conventional FPGA board with DDR4 memory. We conclude that FPGA-based near-memory computing has the potential to alleviate the data movement bottleneck for modern data-intensive applications.

Decomposed bounded floats for fast compression and queries

Modern data-intensive applications often generate large amounts of low precision float data with a limited range of values. Despite the prevalence of such data, there is a lack of an effective solution to ingest, store, and analyze bounded, low-precision, numeric data. To address this gap, we propose Buff, a new compression technique that uses a decomposed columnar storage and encoding methods to provide effective compression, fast ingestion, and high-speed in-situ adaptive query operators with SIMD support.

AnyOLAP

The volume of data that is processed and produced by modern data-intensive applications is constantly increasing. Of course, along with the volume, the interest in analyzing and interpreting this data increases as well. As a consequence, more and more DBMSs and processing frameworks are specialized towards the efficient execution of long-running, read-only analytical queries. Unfortunately, to enable analysis, the data first has to be moved from the source application to the analytics tool via a lengthy ETL process, which increases the runtime and complexity of the analysis pipeline. In this work, we advocate to simply skip ETL altogether. With AnyOLAP, we can perform online analysis of data directly within the source application and while it is running. In the proposed demonstration, the audience will get the chance to put AnyOLAP to the test on a set of data-intensive applications that are supposed to be analyzed while they are up and running. As the entire analysis pipeline of AnyOLAP will be exposed to the audience in form of live and interactive visualizations, users will be able to experience the benefits of true online analysis firsthand.

Can Applications Recover from fsync Failures?

We analyze how file systems and modern data-intensive applications react to fsync failures. First, we characterize how three Linux file systems (ext4, XFS, Btrfs) behave in the presence of failures. We find commonalities across file systems (pages are always marked clean, certain block writes always lead to unavailability) as well as differences (page content and failure reporting is varied). Next, we study how five widely used applications (PostgreSQL, LMDB, LevelDB, SQLite, Redis) handle fsync failures. Our findings show that although applications use many failure-handling strategies, none are sufficient: fsync failures can cause catastrophic outcomes such as data loss and corruption. Our findings have strong implications for the design of file systems and applications that intend to provide strong durability guarantees.

In-Memory Low-Cost Bit-Serial Addition Using Commodity DRAM Technology

In-memory computing architectures present a promising solution to address the memory- and the power-wall challenges by mitigating the bottleneck between processing units and storage. Such architectures incorporate computing functionalities inside memory arrays to make better use of the large internal memory bandwidth, thereby, avoiding frequent data movements. In-DRAM computing architectures offer high throughput and energy improvements in accelerating modern data-intensive applications like machine learning etc . In this manuscript, we propose a vector addition methodology inside DRAM arrays through functional read enabled on local word-lines. The proposed primitive performs majority-based addition operations by storing data in transposed manner. Majority functions are achieved in DRAM cells by activating odd number of rows simultaneously. The proposed majority based bit-serial addition enables huge parallelism and high throughput. We validate the robustness of the proposed in-DRAM computing methodology under process variations to ascertain its reliability. Energy evaluation of the proposed scheme shows 21.7X improvement compared to normal data read operations in standard DDR3-1333 interface. Moreover, compared to state-of-the-art in-DRAM compute proposals, the proposed scheme provides one of the fastest addition mechanisms with low area overhead ( ${k}$ -Nearest Neighbor ( ${k}$ NN) algorithm on the MNIST handwritten digit classification dataset shows 11.5X performance improvement compared to a conventional von-Neumann machine.

Streaming Tables: Native Support to Streaming Data in DBMSs

Data stream management systems (DSMSs) are conceived for running continuous queries (CQs) on the most recently streamed data. This model does not completely fit the needs of several modern data-intensive applications that require to manage recent/historical/static data and execute both CQs and OTQs joining such data. In order to cope with these new needs, some DSMSs have moved toward the integration of database management systems (DBMSs) functionalities to augment their capabilities. In this paper we adopt the opposite perspective and we lay the groundwork for extending DBMSs to natively support streaming facilities. To this end, we introduce a new kind of table, the streaming table , as a persistent structure where streaming data enters and remains stored for a long period, ideally forever. Streaming tables feature a novel access paradigm: continuous writes and one-time as well as continuous reads. We present a streaming table implementation and two novel types of indices that efficiently support both update and scan high rates. A detailed experimental evaluation shows the effectiveness of the proposed technology.

Toggle-Aware Compression for GPUs

Memory bandwidth compression can be an effective way to achieve higher system performance and energy efficiency in modern data-intensive applications by exploiting redundancy in data. Prior works studied various data compression techniques to improve both capacity (e.g., of caches and main memory) and bandwidth utilization (e.g., of the on-chip and off-chip interconnects). These works addressed two common shortcomings of compression: (i) compression/decompression overhead in terms of latency, energy, and area, and (ii) hardware complexity to support variable data size. In this paper, we make the new observation that there is another important problem related to data compression in the context of the communication energy efficiency: transferring compressed data leads to a substantial increase in the number of bit toggles (communication channel switchings from 0 to 1 or from 1 to 0). This, in turn, increases the dynamic energy consumed by on-chip and off-chip buses due to more frequent charging and discharging of the wires. Our results, for example, show that the bit toggle count increases by an average of 2.2× with some compression algorithms across 54 mobile GPU applications. We characterize and demonstrate this new problem across a wide variety of 221 GPU applications and six different compression algorithms. To mitigate the problem, we propose two new toggle-aware compression techniques: energy control and Metadata Consolidation. These techniques greatly reduce the bit toggle count impact of the six data compression algorithms we examine, while keeping most of their bandwidth reduction benefits.

The Collective Index: A Technique for Efficient Processing of Progressive Queries

The emergence of modern data-intensive applications requires sophisticated database techniques for processing advanced types of user queries on massive data. In this paper, we study such a new type of query, called progressive queries. A progressive query is defined as a set of inter-related and incrementally formulated step-queries. A step-query in a progressive query PQ is specified on the fly based on the results of previously-executed step-queries in PQ. Hence, a progressive query cannot be formulated in advance before its execution, which raises challenges for its processing and optimization. We introduce a query model to characterize different types of progressive queries. We then present a new index structure, called the collective index, to efficiently process progressive queries. The collective index technique incrementally evaluates step-queries via dynamically maintained member indexes. Utilizing the special structure of a collective index, the (member) indexes on the input relation(s) of a step-query are efficiently transformed into indexes on the result relation. Algorithms to efficiently process single-input (unary) linear and multiple-input (join) linear progressive queries based on the collective index are presented. Our experiment results show that the proposed collective index technique outperforms the conventional query processing methods in processing progressive queries.