Big Data Query Research Articles

Research Progress of Stream Data Query in Network Space

In recent years, there has been widespread concern about the problems of stream data query both academic and industrial communities. The problems obtained some results. At the same time, big data stream brings great benefits for information society. Information query about stream data form has also brought crucial challenges. However, it is seldom about the research of big data stream query in network space. This paper analyzes the characteristics of stream data query in massive data, discusses the challenges and research issues of data stream for big data query. Finally the works for the data stream query are surveyed.

Research on Big Data Query Online Analysis and Processing Technology on the Basis of Decision Tree Model

Research on Big Data Query Online Analysis and Processing Technology on the Basis of Decision Tree Model

Data processing of high-rate low-voltage distribution grid recordings for smart grid monitoring and analysis

Power networks will change from a rigid hierarchic architecture to dynamic interconnected smart grids. In traditional power grids, the frequency is the controlled quantity to maintain supply and load power balance. Thereby, high rotating mass inertia ensures for stability. In the future, system stability will have to rely more on real-time measurements and sophisticated control, especially when integrating fluctuating renewable power sources or high-load consumers like electrical vehicles to the low-voltage distribution grid. In the present contribution, we describe a data processing network for the in-house developed low-voltage, high-rate measurement devices called electrical data recorder (EDR). These capture units are capable of sending the full high-rate acquisition data for permanent storage in a large-scale database. The EDR network is specifically designed to serve for reliable and secured transport of large data, live performance monitoring, and deep data mining. We integrate dedicated different interfaces for statistical evaluation, big data queries, comparative analysis, and data integrity tests in order to provide a wide range of useful post-processing methods for smart grid analysis. We implemented the developed EDR network architecture for high-rate measurement data processing and management at different locations in the power grid of our Institute. The system runs stable and successfully collects data since several years. The results of the implemented evaluation functionalities show the feasibility of the implemented methods for signal processing, in view of enhanced smart grid operation.

Redoop infrastructure for recurring big data queries

This demonstration presents the Redoop infrastructure, the first full-fledged MapReduce framework with native support for recurring big data queries. Recurring queries, repeatedly being executed for long periods of time over evolving high-volume data, have become a bedrock component in most large-scale data analytic applications. Redoop is a comprehensive extension to Hadoop that pushes the support and optimization of recurring queries into Hadoop's core functionality. While backward compatible with regular MapReduce jobs, Redoop achieves an order of magnitude better performance than Hadoop for recurring workloads. Redoop employs innovative window-aware optimization techniques for such recurring workloads including adaptive window-aware data partitioning, cache-aware task scheduling, and inter-window caching mechanisms. We will demonstrate Redoop's capabilities on a compute cluster against real life workloads including click-stream and sensor data analysis.

Mobile Big Data Query Based on Double R-Tree and Double Indexing

The amount of data in our industry and the world is exploding. Data is being collected and stored at unprecedented rates. The challenge is not only to store and manage the vast volume of data, which is also called big data, but also to analyze and query from it. In order to put forward the universal method to response mobile big data query, queries are separated and grouped according to kinds of query for massive mobile objects in the space. The indexing method for grouping the mobile objects with Grid (GG TPR-tree) has great efficiency to manage a massive capacity of mobile objects within a limited area, but it only could meet a part of requirements for mobile big data query if the GG TPR-tree was used solely. This thesis offers solutions to simple immediate query, simple continuous query, active window query, and continuous window query, dynamic condition query and other query requests by employing DTDI index structure. The experiments prove that with the support of DTDI index structure, query of massive mobile objects has higher precision and better query performance.

Big data challenge: a data management perspective

There is a trend that, virtually everyone, ranging from big Web companies to traditional enterprisers to physical science researchers to social scientists, is either already experiencing or anticipating unprecedented growth in the amount of data available in their world, as well as new opportunities and great untapped value. This paper reviews big data challenges from a data management respective. In particular, we discuss big data diversity, big data reduction, big data integration and cleaning, big data indexing and query, and finally big data analysis and mining. Our survey gives a brief overview about big-data-oriented research and problems.