Support For Queries Research Articles

Design and Implementation of an Efficient Electronic Bank Management Information System Based Data Warehouse and Data Mining Processing

The quantity of electronic bank data grows exponentially with development of Information Technology (IT). The size of these data is impossible for traditional database and human analyst to come up with interesting information that will help in process of decision making. Management Information System (MIS) based Data warehouse (DW) and Data Mining (DM) techniques support the development of IT and process of management decision-making. But the traditional DW size make the query complex, which may cause unacceptable delay in decision support queries. Thus, in this paper an Efficient Electronic Bank MIS based DW and Mining Processing (EEBMIS-DWMP) was developed with cluster and non-cluster indexed view to provide decision-makers with both best response time and precise information. Also, analysis of the multilayer perception neural network, naïve Bayes, random forest, logistic regression, support vector machine and C5.0 on a real-world data of bank was done to improve effectiveness for campaign by analyzing the most useful features that influence campaign success. Results offer how the proposed EEBMIS-DWMP developed bank organizations by comparing performance of system with and without index view in terms of balance accuracy, accuracy, precision, recall, mean absolute error, root mean square error, F measure and running time. Conclusions from results offers that EEBMIS-DWMP can construct a database for each customer, a storage system that integrates data from a variety of sources into a single unified framework, decrease errors and time required to prepare financial reports, quickly access for information, analysis of data in multivariate, accurate prediction of competent, profitability segmentation.

Performance benchmark on semantic web repositories for spatially explicit knowledge graph applications

Knowledge graph has become a cutting-edge technology for linking and integrating heterogeneous, cross-domain datasets to address critical scientific questions. As big data has become prevalent in today's scientific analysis, semantic data repositories that can store and manage large knowledge graph data have become critical in successfully deploying spatially explicit knowledge graph applications. This paper provides a comprehensive evaluation of the popular semantic data repositories and their computational performance in managing and providing semantic support for spatial queries. There are three types of semantic data repositories: (1) triple store solutions (RDF4j, Fuseki, GraphDB, Virtuoso), (2) property graph databases (Neo4j), and (3) an Ontology-Based Data Access (OBDA) approach (Ontop). Experiments were conducted to compare each repository's efficiency (e.g., query response time) in handling geometric, topological, and spatial-semantic related queries. The results show that Virtuoso achieves the overall best performance in both non-spatial and spatial-semantic queries. The OBDA solution, Ontop, has the second-best query performance in spatial and complex queries and the best storage efficiency, requiring the least data-to-RDF conversion efforts. Other triple store solutions suffer from various issues that cause performance bottlenecks in handling spatial queries, such as inefficient memory management and lack of proper query optimization.

A novel file system supporting rich file classification

Existing implementations of file systems often seem to be made on an ad hoc and implicit basis. This paper aims to enhance the organization of files and retrieval of files by modifying the traditional hierarchical file system to improve built-in query support and bulk metadata updates supported at the file system level. We introduce tags in a hierarchy of file collections and use links to allow file retrieval from multiple paths as files exist in multiple directories simultaneously. By using a series of modest changes to the hierarchical file system, we propose a novel Linked Tree Tags (LTTs) model. These changes include using multiple tags instead of names, collections instead of directories, exposing a query language at the Application Programming Interface (API) level, and allowing controlled file links. We assess our model's expressive capability and demonstrate that LTTs overcome traditional file systems' limits and provide users with the to manage their files easily.

Formula omitted]: A global optimization algorithm to solve the view selection problem for large-scale workload optimization

In advanced database design such as Big Data Warehouses, optimizing large-scale decision support queries passes through the selection of redundant structures such as materialized views and indexes. Materialized View Selection (MVS) is one of the most studied problems in the context of the physical design of advanced databases. It is known as an NP-Hard problem. Several algorithms have been proposed to find, within a reasonable computation time, the appropriate materialized views that reduce as much as possible the query processing cost and the view maintenance cost w.r.t. a storage constraint. By analyzing the state-of-art studies, we figure out that almost all are workload-driven approaches. Their efficiency strongly depends on the number of queries of the considered workload. These approaches manage a small set of queries. Having efficient algorithms for selecting materialized views based on a very large set of queries has become a crucial issue for advanced applications. In this paper, we proposed a ZigZag+ approach that uses the multiple view processing plan (MVPP) as a basic data structure that unifies all query plans. Due to the no-unicity of this MVPP, our approach aims at exploring all possible MVVPs, and for each exploration, it identifies materialized views. After this selection, update operations of the current MVPP are performed. Zigzagging from one MVPP to another is guided by the quality of selected views. Intensive experiments have been conducted to evaluate the effectiveness and efficiency of our proposal by considering large workloads and comparing it with state-of-art approaches.

Speculative Multi-Level Access in LSM Tree-Based KV Store

The log-structured merge (LSM) tree has been widely used due to its small DRAM footprint, easy range query support, and write-friendly characteristics. However, it has an inherent problem of read amplification; a single key lookup operation can be accompanied by multiple disk read operations which prolong the latency of lookup operation and fail to exploit the parallelism of SSDs. This paper proposes a speculative multi-level access scheme, which accesses multiple levels simultaneously and speculatively. By accessing multiple levels together, it can exploit multi-I/O-handling capability of the SSD and also overlap the execution of CPU with I/O device, thereby improving the performance of lookup operations. Our evaluation results demonstrate that the proposed scheme improves the throughput by up to 85% and reduces the average latency by up to 45% with realistic key-value workloads.

Unified many-worlds browsing of arbitrary physics-based animations

Manually tuning physics-based animation parameters to explore a simulation outcome space or achieve desired motion outcomes can be notoriously tedious. This problem has motivated many sophisticated and specialized optimization-based methods for fine-grained (keyframe) control, each of which are typically limited to specific animation phenomena, usually complicated, and, unfortunately, not widely used. In this paper, we propose Unified Many-Worlds Browsing (UMWB), a practical method for sample-level control and exploration of physics-based animations. Our approach supports browsing of large simulation ensembles of arbitrary animation phenomena by using a unified volumetric WORLDPACK representation based on spatiotemporally compressed voxel data associated with geometric occupancy and other low-fidelity animation state. Beyond memory reduction, the WORLDPACK representation also enables unified query support for interactive browsing: it provides fast evaluation of approximate spatiotemporal queries, such as occupancy tests that find ensemble samples ("worlds") where material is either IN or NOT IN a user-specified spacetime region. WORLDPACKS also support real-time hardware-accelerated voxel rendering by exploiting the spatially hierarchical and temporal RLE raster data structure. Our UMWB implementation supports interactive browsing (and offline refinement) of ensembles containing thousands of simulation samples, and fast spatiotemporal queries and ranking. We show UMWB results using a wide variety of physics-based animation phenomena---not just JELL-O ® .

Strategies for the Storage of Large LiDAR Datasets—A Performance Comparison

The widespread use of LiDAR technologies has led to an ever-increasing volume of captured data that pose a continuous challenge for its storage and organization, so that it can be efficiently processed and analyzed. Although the use of system files in formats such as LAS/LAZ is the most common solution for LiDAR data storage, databases are gaining in popularity due to their evident advantages: centralized and uniform access to a collection of datasets; better support for concurrent retrieval; distributed storage in database engines that allows sharding; and support for metadata or spatial queries by adequately indexing or organizing the data. The present work evaluates the performance of four popular NoSQL and relational database management systems with large LiDAR datasets: Cassandra, MongoDB, MySQL and PostgreSQL. To perform a realistic assessment, we integrate these database engines in a repository implementation with an elaborate data model that enables metadata and spatial queries and progressive/partial data retrieval. Our experimentation concludes that, as expected, NoSQL databases show a modest but significant performance difference in favor of NoSQL databases, and that Cassandra provides the best overall database solution for LiDAR data.

Estimate the Performance of Cloudera Decision Support Queries

Hive and Impala queries are used to process a big amount of data. The overwriting amount of information requires an efficient data processing system. When we deal with a long-term batch query and analysis Hive will be more suitable for this query. Impala is the most powerful system suitable for real-time interactive Structured Query Language (SQL) query which are added a massive parallel processing to Hadoop distributed cluster. The data growth makes a problem with SQL Cluster because the execution processing time is increased. In this paper, a comparison is demonstrated between the performance time of Hive, Impala and SQL on two different data models with different queries chosen to test the performance. The results demonstrate that Impala outperforms Hive and SQL cluster when it comes to analyze data and processing tasks. Using two benchmark datasets, TPC-H and statistical computing, we compare the performance of Hive, Impala, and SQL clusters 2009 Statistical Graphics Data Expo.

Non-Interactive Multi-Client Searchable Encryption: Realization and Implementation

In this article, we introduce a new mechanism for constructing multi-client searchable encryption (SE). By tactfully leveraging the RSA-function, we propose the first multi-client SE protocol that successfully avoids per-query interaction between data owner and client. Therefore, our approach significantly reduces the communication cost by eliminating the need for data owner to authorize client queries at all times. To be compatible with the RSA-based approach, we also present a deterministic and memory-efficient &#x2018;keyword to prime&#x2019; hash function, which may be of independent interest. Further, to improve efficiency, we put forward a more generic construction from set-constrained PRFs. The construction not only inherits the merits of our first protocol, but also achieves an enhanced security (against untrusted clients), where colluding attack among clients is also taken into account. Both protocols are instantiated via the recent representative SE protocol by Cash <i>et al.</i> with the support of boolean queries. At last, we implement our proposed protocols and comprehensively evaluate their performance to demonstrate their practicability and scalability.

Future-Based Persistent Spatial Data Structure for NVM-Based Manycore Machines

R-trees have been popular for their support of multidimensional data and high-performing queries. FBR-tree is the state-of-the-art concurrent variant of the R-tree for Intel DC Persistent Memory (DCPM). However, its adoption on manycore servers is impeded by concurrency limitations due to lengthy, lock-based synchronization, including structure modification operations, split and merge. Additionally, emerging DCPM-based machines are equipped with multiple CPU sockets, forming a non-uniform memory access (NUMA) architecture. FBR-tree’s lack of NUMA-awareness induces further performance overhead from remote memory accesses. In this paper, we propose MPR-tree, a concurrent, NUMA-aware and persistent future-based R-tree for DCPM servers. MPR-tree focuses on insert operations due to their laborious nature. MPR-tree relies on per-thread local future objects and a global R-tree. To introduce NUMA-awareness and minimize remote memory accesses, MPR-tree adopts per-socket dedicated asynchronous evaluate threads to checkpoint future objects to the global R-tree. MPR-tree employs an in-memory hash table to mitigate the read overhead of key searches over the future objects. We implemented MPR-tree atop FBR-tree and evaluated its performance on a server with 40 physical cores for insert and lookup queries, and it showed that MPR-tree outperforms FBR-tree on average by 2× on log10 scale.

A forward and backward secure conjunctive keywords dynamic searchable encryption scheme

Dynamic searchable symmetric encryption (DSSE) allows a user to carryout search and update operations on the outsourced encrypted data on cloud. The recent research work in DSSE exposed many flaws in existing strategies by introducing leakage abuse attacks. To protect DSSE schemes from leakage, forward and backward privacy notions were introduced. In this paper, we propose a forward and backward secure chained dictionary-based indexing mechanism with support for conjunctive keyword queries. The proposed scheme uses a minimal leakage index design, requiring compact storage at client, easier update operations at server and removes entries of deleted files from index to ensure backward privacy resulting in reduced index size. It ensures that the searches followed by update operation do not violate query privacy. Rigorous security analysis and experimental evaluations show that our scheme achieves sub-linear search time and supports efficient update operations ensuring forward and backward privacy.

Query Processing in Blockchain Systems: Current State and Future Challenges

When, in 2008, Satoshi Nakamoto envisioned the first distributed database management system that relied on cryptographically secured chain of blocks to store data in an immutable and tamper-resistant manner, his primary use case was the introduction of a digital currency. Owing to this use case, the blockchain system was geared towards efficient storage of data, whereas the processing of complex queries, such as provenance analyses of data history, is out of focus. The increasing use of Internet of Things technologies and the resulting digitization in many domains, however, have led to a plethora of novel use cases for a secure digital ledger. For instance, in the healthcare sector, blockchain systems are used for the secure storage and sharing of electronic health records, while the food industry applies such systems to enable a reliable food-chain traceability, e.g., to prove compliance with cold chains. In these application domains, however, querying the current state is not sufficient—comprehensive history queries are required instead. Due to these altered usage modes involving more complex query types, it is questionable whether today’s blockchain systems are prepared for this type of usage and whether such queries can be processed efficiently by them. In our paper, we therefore investigate novel use cases for blockchain systems and elicit their requirements towards a data store in terms of query capabilities. We reflect the state of the art in terms of query support in blockchain systems and assess whether it is capable of meeting the requirements of such more sophisticated use cases. As a result, we identify future research challenges with regard to query processing in blockchain systems.

Towards Multi-user Searchable Encryption Scheme with Support for SQL Queries

Towards Multi-user Searchable Encryption Scheme with Support for SQL Queries

Spatial Data Sequence Selection Based on a User-Defined Condition Using GPGPU

The size of spatial data is growing intensively due to the emergence of and the tremendous advances in technology such as sensors and the internet of things. Supporting high-performance queries on this large volume of data becomes essential in several data- and compute-intensive applications. Unfortunately, most of the existing methods and approaches are based on a traditional computing framework (uniprocessors) which makes them not scalable and not adequate to deal with large-scale data. In this work, we present a high-performance query for massive spatio–temporal data. The query consists of selecting fixed size raster subsequences, based on the average of their region of interest, from a spatio–temporal raster sequence satisfying a user threshold condition. In our paper, for the purpose of simplification, we consider that the region of interest is the entire raster and not only a subregion. Our aim is to speed up the execution using parallel primitives and pure CUDA. Furthermore, we propose a new method based on a sorting step to save computations and boost the speed of the query execution. The test results show that the proposed methods are faster and good performance is achieved even with large-scale rasters and data.

QRB-tree Indexing: Optimized Spatial Index Expanding upon the QR-tree Index

Support for region queries is crucial in geographic information systems, which process exact queries through spatial indexing to filter features and subsequently refine the selection. Although the filtering step has been extensively studied, the refinement step has received little attention. This research builds upon the QR-tree index, which decomposes space into hierarchical grids, registers features to the grids, and builds an R-tree for each grid, to develop a new QRB-tree index with two levels of optimization. In the first level, a bucket is introduced in every grid in the QR-tree index to accelerate the loading and search steps of a query region for the grids within the query region. In the second level, the number of candidate features to be eliminated is reduced by limiting the features to those registered to the grids covering the corners of the query region. Subsequently, an approach for determining the maximal grid level, which significantly affects the performance of the QR-tree index, is proposed. Direct comparisons of time costs with the QR-tree index and geohash index show that the QRB-tree index outperforms the other two approaches for rough queries in large query regions and exact queries in all cases.

Few‐shot learning with relation propagation and constraint

Previous deep learning methods usually required large-scale annotated data, which is computationally exhaustive and unrealistic in certain scenarios. Therefore, few-shot learning, where only a few annotated training images are available for training, has attracted increasing attention these days, showing huge potential in practical applications, such as portable equipment or security inspection, and so on. However, current few-shot learning methods usually neglect the valuable semantic correlations between samples, thereby failing in extracting discriminating relations to achieve accurate predictive results. In this work, extending on a recent state-of-the-art few-shot learning method, transductive relation-propagation network (TRPN), which considers the correlations between training samples, a constrained relation-propagation network is proposed to further regularise the distilled correlations and thus achieve favourable few-shot classification performance. The proposed framework contains three main components, namely preprocess module, relational propagation module, and relation constraint module. First, sample features are extracted and a relation graph node is constructed by treating the relation of each support–query pair as a graph node in the preprocess module. After that, in the relation propagation module (RPM), the valuable information of support–query pairs is modelled and propagated to directly generate the relational representations for further prediction. Then, a relation constraint module is introduced to regularise the relational representations and make it consistent with the ground-truth relations as much as possible. With the guidance of the effective RPM and relation constraint module, the relational representations of the support–query pairs are distinguishable and thus can achieve accurate predictive results. Comprehensive experiments conducted on widely used benchmarks validate the effectiveness of our method compared to state-of-the-art few-shot classification approaches.

Lock-free Contention Adapting Search Trees

Concurrent key-value stores with range query support are crucial for the scalability and performance of many applications. Existing lock-free data structures of this kind use a fixed synchronization granularity. Using a fixed synchronization granularity in a concurrent key-value store with range query support is problematic as the best performing synchronization granularity depends on a number of factors that are difficult to predict, such as the level of contention and the number of items that are accessed by range queries. We present the first linearizable lock-free key-value store with range query support that dynamically adapts its synchronization granularity. This data structure is called the lock-free contention adapting search tree (LFCA tree). An LFCA tree automatically performs local adaptations of its synchronization granularity based on heuristics that take contention and the performance of range queries into account. We show that the operations of LFCA trees are linearizable, that the lookup operation is wait-free, and that the remaining operations (insert, remove and range query) are lock-free. Our experimental evaluation shows that LFCA trees achieve more than twice the throughput of related lock-free data structures in many scenarios. Furthermore, LFCA trees are able to perform substantially better than data structures with a fixed synchronization granularity over a wide range of scenarios due to their ability to adapt to the scenario at hand.

Dragoon: a hybrid and efficient big trajectory management system for offline and online analytics

With the explosive use of GPS-enabled devices, increasingly massive volumes of trajectory data capturing the movements of people and vehicles are becoming available, which is useful in many application areas, such as transportation, traffic management, and location-based services. As a result, many trajectory data management and analytic systems have emerged that target either offline or online settings. However, some applications call for both offline and online analyses. For example, in traffic management scenarios, offline analyses of historical trajectory data can be used for traffic planning purposes, while online analyses of streaming trajectories can be adopted for congestion monitoring purposes. Existing trajectory-based systems tend to perform offline and online trajectory analysis separately, which is inefficient. In this paper, we propose a hybrid and efficient framework, called Dragoon, based on Spark, to support both offline and online big trajectory management and analytics. The framework features a mutable resilient distributed dataset model, including RDD Share, RDD Update, and RDD Mirror, which enables hybrid storage of historical and streaming trajectories. It also contains a real-time partitioner capable of efficiently distributing trajectory data and supporting both offline and online analyses. Therefore, Dragoon provides a hybrid analysis pipeline. Support for several typical trajectory queries and mining tasks demonstrates the flexibility of Dragoon. An extensive experimental study using both real and synthetic trajectory datasets shows that Dragoon (1) has similar offline trajectory query performance with the state-of-the-art system UlTraMan; (2) decreases up to doubled storage overhead compared with UlTraMan during trajectory editing; (3) achieves at least 40% improvement of scalability compared with popular streaming processing frameworks (i.e., Flink and Spark Streaming); and (4) offers an average doubled performance improvement for online trajectory data analytics.

BciPy: brain–computer interface software in Python

ABSTRACT There are high technological and software demands associated with conducting Brain–Computer Interface (BCI) research. In order to accelerate the development and accessibility of BCIs, it is worthwhile to focus on open-source and community desired tooling. Python, a prominent computer language, has emerged as a language of choice for many research and engineering purposes. In this article, BciPy, an open-source, Python-based software for conducting BCI research is presented. It was developed with a focus on restoring communication using Event-Related Potential (ERP) spelling interfaces; however, it may be used for other non-spelling and non-ERP BCI paradigms. Major modules in this system include support for data acquisition, data queries, stimuli presentation, signal processing, signal viewing and modeling, language modeling, task building, and a simple Graphical User Interface (GUI).

Array databases: concepts, standards, implementations

Multi-dimensional arrays (also known as raster data or gridded data) play a key role in many, if not all science and engineering domains where they typically represent spatio-temporal sensor, image, simulation output, or statistics “datacubes”. As classic database technology does not support arrays adequately, such data today are maintained mostly in silo solutions, with architectures that tend to erode and not keep up with the increasing requirements on performance and service quality. Array Database systems attempt to close this gap by providing declarative query support for flexible ad-hoc analytics on large n-D arrays, similar to what SQL offers on set-oriented data, XQuery on hierarchical data, and SPARQL and CIPHER on graph data. Today, Petascale Array Database installations exist, employing massive parallelism and distributed processing. Hence, questions arise about technology and standards available, usability, and overall maturity. Several papers have compared models and formalisms, and benchmarks have been undertaken as well, typically comparing two systems against each other. While each of these represent valuable research to the best of our knowledge there is no comprehensive survey combining model, query language, architecture, and practical usability, and performance aspects. The size of this comparison differentiates our study as well with 19 systems compared, four benchmarked to an extent and depth clearly exceeding previous papers in the field; for example, subsetting tests were designed in a way that systems cannot be tuned to specifically these queries. It is hoped that this gives a representative overview to all who want to immerse into the field as well as a clear guidance to those who need to choose the best suited datacube tool for their application. This article presents results of the Research Data Alliance (RDA) Array Database Assessment Working Group (ADA:WG), a subgroup of the Big Data Interest Group. It has elicited the state of the art in Array Databases, technically supported by IEEE GRSS and CODATA Germany, to answer the question: how can data scientists and engineers benefit from Array Database technology? As it turns out, Array Databases can offer significant advantages in terms of flexibility, functionality, extensibility, as well as performance and scalability—in total, the database approach of offering “datacubes” analysis-ready heralds a new level of service quality. Investigation shows that there is a lively ecosystem of technology with increasing uptake, and proven array analytics standards are in place. Consequently, such approaches have to be considered a serious option for datacube services in science, engineering and beyond. Tools, though, vary greatly in functionality and performance as it turns out.