Efficiency Of Query Processing Research Articles

Constructing renewable energy sphere for efficient search

In the field of renewable energy (RE), traditional methods for understanding, development, and education have heavily relied on 2D table formats for data retrieval and presentation. However, these formats suffer from inefficiency and an inability to fully capture the relationships between variables. In response to this challenge, Li et al. (2022) demonstrated that their therapeutic 3D sphere system significantly outperformed the therapeutic 2D table in query processing time. Furthermore, Shih et al. (2020) utilized a T-test to underscore the superiority of the TED Talks sphere over the TED Talks 2D table format website in terms of convenience, query processing efficiency, usefulness, and intuitiveness. Building upon this foundation, this study referenced and enhanced the work of Li et al. by introducing a three-dimensional spherical visualization model driven by cluster-based binary integer programming (BIP). This novel approach resulted in a 58% reduction in information retrieval time and achieved an impressive 86.4% user satisfaction rating. It is worth noting that compared to the previous Systematic Review and Synthesis (SRS), user satisfaction experienced a significant 19% improvement. This improvement enhances the search efficiency for RE information, thereby fostering public awareness and development of RE. Taking energy supply-demand balance sheets as an example, this study demonstrates that the spherical retrieval system (SRS) constructed by the visualization 3D spherical model resolves the issue of inefficient retrieval in 2D table formats, and the newly added functionality of highlighting objects on the spherical surface intuitively illustrates the one-to-many relationships between individual RE supply and demand. To optimize hardware and software performance, data mining techniques are employed to cluster supply data, reducing complexity, and improving optimization efficiency. The experiment is that testers search for the same targets and compared the proposed 3D SRS with the Taiwan Power Company (Taipower) 2D website in terms of retrieval speed and accuracy. The statistical results of the T-test indicate that the proposed 3D SRS is more effective and accurate than the Taipower 2D website. The research findings indicate: 1. The new 3D SRS reduces query time by 58%. 2. Adopting the k-means algorithm for clustering and batch optimization can transform from an initially infeasible solution to finding the optimal solution within 12 s. This method also reduces computer software and hardware costs, providing effective solutions for relevant applications. These technologies can be used for the education, promotion, publicity, and dissemination of RE.

Query Optimization by Quantifier Elimination

Query optimizers have a limited arsenal of techniques for optimizing nested queries. In this paper, we develop a new approach for query optimization based on quantifier elimination. Quantifier elimination is a well-established tool for proving the decidability of logical theories. Here, however, we show that it can be turned into an effective query optimization technique that may yield asymptotic improvements in query processing efficiency. In addition, the technique establishes a foundation for certain well-known but previously little-understood aggregation based techniques for optimizing nested queries.

A learned cost model for big data query processing

The efficiency of query processing in the Spark SQL big data processing engine is significantly affected by execution plans and allocated resources. However, existing cost models for Spark SQL rely on hand-crafted rules. While learning-based cost models have been proposed for relational databases, they do not consider available resources. To address this issue, we propose a resource-aware deep learning model capable of automatically predicting query plan execution times based on historical data. To train our model, we embed query execution plans within a query plan tree and extracted features from allocated resources. An adaptive attention mechanism is integrated into the deep learning model to enhance prediction accuracy. Additionally, we extract sufficient features to represent data information and learn the effect of the data on query execution. This approach reduces the need for model retraining owing to data changes. The experimental results demonstrate that our deep cost model outperforms traditional rule-based methods and relational database learning-based optimizers in predicting query plan execution times.

Lemo: A Cache-Enhanced Learned Optimizer for Concurrent Queries

With the expansion of modern database services, multi-user access has become a crucial feature in various practical application scenarios, including enterprise applications and e-commerce platforms. However, if multiple users submit queries within a short time frame, it can result in potential issues such as redundant computation and query concurrency. Unfortunately, most existing multi-query optimization methods, which aim to enhance query processing efficiency, have not adequately addressed these two problems, especially in the setting where multiple queries are being executed concurrently. To this end, we propose a novel method named Lemo for the multi-query optimization problem. Specifically, we propose a novel value network to predict latencies of concurrent queries as the foundation model for query plan generation. Furthermore, we introduce a shared buffer manager component to cache the intermediate results of sub-queries. The shared buffer manager applies a novel replacement policy to maintain the cached buffer with the objective of maximizing the opportunity for the reuse of the cached sub-queries. Based on the shared buffer, our proposed value network can incorporate the cached results into cost estimation to further guide Lemo in generating query plans, thus avoiding redundant computation. Lemo has been integrated into PostgreSQL and experiments conducted on real datasets with PostgreSQL show that it outperforms all the baselines in efficiency.

A PID-Based kNN Query Processing Algorithm for Spatial Data.

As a popular spatial operation, the k-Nearest Neighbors (kNN) query is widely used in various spatial application systems. How to efficiently process a kNN query on spatial big data has always been an important research topic in the field of spatial data management. The centralized solutions are not suitable for spatial big data due to their poor scalability, while the existing distributed solutions are not efficient enough to meet the high real-time requirements of some spatial applications. Therefore, we introduce the Proportional Integral Derivative (PID) control technology into kNN query processing and propose a PID-based kNN query processing algorithm (PIDKNN) for spatial big data based on Spark. In this algorithm, the whole data space is divided into grid cells of the same size using the grid partition method, and the grid-based index is constructed. On this basis, the grid-based density peak clustering algorithm is used to cluster spatial data, and the corresponding PID parameters are set for each cluster. When performing kNN queries, the PID algorithm is used to estimate the radius growth step size of kNN queries, thereby realizing kNN query processing with a variable query radius growth step based on a feedback mechanism, which greatly improves the efficiency of kNN query processing. A series of experimental results show that the PIDKNN algorithm has good performance and scalability and is superior to the existing parallel kNN query processing methods.

Privacy-Preserving Keyword Similarity Search Over Encrypted Spatial Data in Cloud Computing

With the proliferation of cloud computing, data owners can outsource the spatial data from the Internet of Things devices to a cloud server to enjoy the pay-as-you-go storage resources and location-based services. However, the outsourced services may raise privacy concerns, since the cloud server may not be fully trusted for both data owners and search users. If the data owners and search users conventionally encrypt the spatial data and query requests, the efficiency and functionality of query processing are weakened. Most of the existing works only focus on spatial data search or keyword search and do not consider spatial keyword search over encrypted data. In this article, we first design a geometric range query (GRQ) scheme, which can generate an arbitrary geometric range to fit the search user’s desired spatial data while protecting location privacy. Furthermore, based on GRQ, we propose a multidimensional spatial keyword similarity search scheme with access control (MSSAC) by integrating the polynomial function and matrix transformation. Specifically, an access control strategy is defined by a role-based polynomial function, which is embedded in the vectors of indices and trapdoors to achieve efficient and lightweight access control. Moreover, MSSAC enables the cloud server to execute compute-then-compare operations for spatial keyword search in a privacy-preserving manner by leveraging techniques of randomizable permutation and matrix multiplication. The formal security analyses and extensive experiments demonstrate that GRQ and MSSAC preserve the privacy of data owners and search users while achieving efficient spatial keyword search.

Effective and efficient skyline query processing over attribute-order-preserving-free encrypted data in cloud-enabled databases

Making co-existent and convergent the need for efficiency of relational query processing over Clouds and the security of data themselves is figuring-out how one of the most challenging research problems in the Big Data era. Indeed, in actual analytics-oriented engines, such as Google Analytics and Amazon S3, where key–value storage-representation and efficient-management models are employed as to cope with the simultaneous processing of billions of transactions, querying encrypted data is becoming one of the most annoying problem, which has also attracted a great deal of attention from the research community. While this issue has been applied to a large variety of data formats, e.g. relational, RDF and multidimensional data, very few initiatives have pointed-out skyline query processing over encrypted data, which is, indeed, relevant for database analytics. In order to fulfill this methodological and technological gap, in this paper we introduce an innovative algorithm for effectively and efficiently supporting skyline query processing over encrypted data in Cloud-enabled databases, named as Attribute-Order-Preserving-Free-SFS (AOPF-SFS), a suitable extension of the well-known Sort-Filter-Skyline (SFS) algorithm. The proposed algorithm enables the processing of skyline queries over encrypted data, even without preserving the order on each attribute as order-preserving encryption would do. We also present eSkyline, a prototype system that embeds AOPF-SFS equipped with a suitable query interface comprising an encryption scheme that facilitates the evaluation of domination relationships, hence allows for state-of-the-art skyline processing algorithms to be used. In order to prove the effectiveness and the reliability of our system, we also provide the details of the underlying encryption scheme, plus a suitable GUI that allows a user to interact with a server, and showcases the efficiency of computing skyline queries and decrypting the results.

Exploring cohesive subgraphs with vertex engagement and tie strength in bipartite graphs

We propose a novel cohesive subgraph model called τ -strengthened ( α , β ) -core (denoted as ( α , β ) τ -core), which is the first to consider both tie strength and vertex engagement on bipartite graphs . An edge is a strong tie if contained in at least τ butterflies ( 2 × 2 -bicliques). ( α , β ) τ -core requires each vertex on the upper or lower level to have at least α or β strong ties, given strength level τ . To retrieve the vertices of ( α , β ) τ -core optimally, we construct index I α , β , τ to store all ( α , β ) τ -cores. Effective optimization techniques are proposed to improve index construction . To make our idea practical on large graphs, we propose 2D-indexes I α , β , I β , τ , and I α , τ that selectively store the vertices of ( α , β ) τ -core for some α , β , and τ . The 2D-indexes are more space-efficient and require less construction time, each of which can support ( α , β ) τ -core queries. As query efficiency depends on input parameters and the choice of 2D-index, we propose a learning-based hybrid computation paradigm by training a feed-forward neural network to predict the optimal choice of 2D-index that minimizes the query time. Extensive experiments show that (1) ( α , β ) τ -core is an effective model capturing unique and important cohesive subgraphs; (2) the proposed techniques significantly improve the efficiency of index construction and query processing .

Improving the Performance of Multidimensional Clinical Data for OLAP using an Optimized Data Clustering approach

Medicine is a fresh way to utilize for curing, analyzing and detecting the diseases through data clustering with OLAP (Online Analytical Processing). The large amount of multidimensional clinical data is reduced the efficiency of OLAP query processing by enhancing the query accessing time. Hence, the performance of OLAP model is improved by using data clustering in which huge data is divided into several groups (clusters) with cluster heads to achieve fast query processing in least time. In this paper, a Dragon Fly Optimization based Clustering (DFOC) approach is proposed to enhance the efficiency of data clustering by generating optimal clusters from multidimensional clinical data for OLAP. The results are evaluated on MATLAB 2019a tool and shown the better performance of DFOC against other clustering methods ACO, GA and K-Means in terms of intra-cluster distance, purity index, F-measure, and standard deviation

Public Sentiment Big Data Query Processing and Optimization with Unified Storage of Source and Meta Data

Public sentiment big data has the characteristics of mass, multi-source, heterogeneity and multi-mode. At present, it mostly adopts separate storage strategies and has low storage and query efficiency. To this end, a distributed storage model is proposed in which structured and unstructured data correspond and are stored uniformly. By establishing a unified storage framework for source metadata, the original source data and characteristic metadata are stored uniformly according to different storage methods. Based on the above storage architecture, a hierarchical index structure is proposed to improve the efficiency of big data query. The unified storage model of source metadata is compared with the common storage methods, and is in the leading position in terms of Block number and query processing efficiency.

View Materialization Over Big Data

Advances in technology have resulted in the generation of a large volume of heterogeneous big data for large enterprises engaged in e-commerce, healthcare, education, etc. This is being created at a rapid rate but is low in its veracity. This big data includes large sets of semi-structured and unstructured data and is stored over a distributed file system (DFS). This data can be processed in a fault tolerant manner using several frameworks, tools, and advanced database technologies. Big data can provide important information, which can be used for business decision making. View materialization, which has been widely studied for structured databases or data warehouse, has been extended to big data to enhance efficiency of big data query processing. This paper focuses on the selection of big data views for materialization. The big data views can be identified by extracting a set of query attributes from the set of query workload of an enterprise. The query attributes are interrelated resulting in the creation of alternate access paths for query evaluation. The cost of query processing using big data views involves the integrity of different data types of heterogeneous big data, frequency of queries, change in the size of big data, selected sets of big data materialized views, and updates on big data and these sets of materialized views. The cost of query processing is computed using the stored size of big data views on the DFS system, which is a consistent processing framework of DFS. A big data view selection algorithm that is capable of selecting views from structured, semi-structured, and unstructured data has been proposed in this paper. The proposed algorithm would select big data views that would result in faster processing of most user queries resulting in efficient decision making.

Enhanced query processing over semantic cache for cloud based relational databases

Semantic cache diminishes the expectancies of data retrieval over distributed system like cloud-based systems, by reusing already extracted data. It improves the performance of the retrieval system with limited bandwidth which is a key requirement in cloud, fog, edge and other mobile computing technologies. Cache management and query processing over semantic cache are two key activities which should be handled carefully. Query dispensation performs a significant part in terms of reusability in already retrieved data efficiently. Competent query dispensation algorithms made the semantic cache more efficient. This paper identifies four issues that cause decreasing the efficiency of query processing algorithm. The solution is provided to the identified issues to improve the efficiency of query processing. Improvement in efficiency is discussed along each of the issues. Efficiency analysis for enhanced algorithm is also provided and compared with state-of-the-art algorithms in the field. The proposed approach significantly improves the efficiency in many ways.

A New Methodology for Computing Semantic Relatedness: Modified Latent Semantic Analysis by Fuzzy Formal Concept Analysis

Semantic similarity is used to perceive the meaning of words in textual data and having several applications in the field of Computational linguistic and Natural language processing. Semantic similarity and semantic relatedness are interchangeable terms, in Semantic relatedness all types of semantic relationships are considered i.e. (has, part-of, contains etc.) whereas in semantic similarity only “is-a” type of relationship is used. Computing accurate similarity improves the efficiency of query processing and understanding of textual data more proficiently. In this work, a novel hybrid approach of corpus based and knowledge based method is proposed approach to compute the semantic relatedness among words. In this hybrid method, Latent semantic analysis is used to find the concepts and their correlation value and mapping is done for different words with the help of fuzzy formal concept analysis.To compute semantic relatedness on different words a fuzzy set similarity measure is used.The proposed approach has been evaluated on solar domain and attains improved results as compared to other baseline measures. This method can be used for any doamin as it is a generalized approach for similarity computation.

Materialized View Selection for Query Performance Enhancement Using Stochastic Ranking Based Cuckoo Search Algorithm

Materialized view selection (MVS) improves the query processing efficiency and performance for making decisions effectively in a data warehouse. This problem is NP-hard and constrained optimization problem which involves space and cost constraint. Various optimization algorithms have been proposed in literature for optimal selection of materialized views. Few works exist for handling the constraints in MVS. In this study, authors have proposed the Cuckoo Search Algorithm (CSA) for optimization and Stochastic Ranking (SR) for handling the constraints in solving the MVS problem. The motivation behind integrating CS with SR is that fewer parameters have to be fine tuned in CS algorithm than in genetic and Particle Swarm Optimization (PSO) algorithm and the ranking method of SR handles the constraints effectively. For proving the efficiency and performance of our proposed algorithm Stochastic Ranking based Cuckoo Search Algorithm for Materialized View Selection (SRCSAMVS), it has been compared with PSO, genetic algorithm and the constrained evolutionary optimization algorithm proposed by Yu et al. SRCSAMVS outperforms in terms of query processing cost and scalability of the problem.

Scaling the Construction of Wavelet Synopses for Maximum Error Metrics

Modern analytics involve computations over enormous numbers of data records. The volume of data and the stringent response-time requirements place increasing emphasis on the efficiency of approximate query processing. A major challenge over the past years has been the construction of synopses that provide a deterministic quality guarantee, often expressed in terms of a maximum error metric. By approximating sharp discontinuities, wavelet decomposition has proved to be a very effective tool for data reduction. However, existing wavelet thresholding schemes that minimize maximum error metrics are constrained with impractical complexities for large datasets. Furthermore, they cannot efficiently handle the multi-dimensional version of the problem. In order to provide a practical solution, we develop parallel algorithms that take advantage of key-properties of the wavelet decomposition and allocate tasks to multiple workers. To that end, we present (i) a general framework for the parallelization of existing dynamic programming algorithms, (ii) a parallel version of one such DP algorithm, and (iii) two highly efficient distributed greedy algorithms that can deal with data of arbitrary dimensionality. Our extensive experiments on both real and synthetic datasets over Hadoop show that the proposed algorithms achieve linear scalability and superior running-time performance compared to their centralized counterparts.

A new Top-k query processing algorithm to guarantee confidentiality of data and user queries on outsourced databases

With the growth of cloud computing, database outsourcing has attracted a lot of interest. Due to the serious privacy threats in cloud computing, databases need to be encrypted before being outsourced to the cloud. Therefore, various Top-k query processing algorithms have been studied for encrypted databases. However, existing algorithms are either insecure or inefficient. Therefore, in this paper we propose a new Top-k query processing algorithm that guarantees confidentiality of both data and user queries while hiding data access patterns. Our algorithm also allows the query issuer to not participate in the query processing. To achieve a high query processing efficiency, we implement new secure protocols using Yao’s garbled circuit and a data packing technique. A performance analysis shows that the proposed algorithm outperforms the existing work in terms of query processing costs.

LB-Grid: An SSD efficient Grid File

Recent advances in non-volatile memory technology have led to the introduction of solid state drives (SSD). NVMe SSDs are the latest development in flash based solid state drives and they were designed as a means of low latency and high bandwidth. Many research studies seek for taking advantage of this new technology to accelerate data management. Multidimensional indexes are fundamental for the efficiency of spatial query processing. In this work, we study the implication of high performance NVMe drives on spatial indexing. More specifically, we present an in-depth performance analysis of the Grid File in flash storage and we introduce LB-Grid, a write efficient variant of Grid File for flash based solid state drives. We present new query algorithms for both LB-Grid and Grid File that exploit the high internal parallelism and I/O bandwidth of NVMe SSDs. Experimental results unveil the efficiency of the proposed algorithms. Utilizing a test set of 500M points, LB-Grid appears to be up to 2.26 times faster than Grid File, up to 5.5 times faster than the R∗-tree, and up to 3.3 times faster than the FAST-Rtree in update intensive workloads. On the other hand, the Grid File presents better performance in read intensive workloads; exploiting a batch read operation, it achieves a speedup up to 10.2x in range queries, up to 1.56x in kNN and 4.6x in group point queries.

Privacy-Preserving Multi-Keyword Top- Similarity Search Over Encrypted Data

Cloud computing provides individuals and enterprises massive computing power and scalable storage capacities to support a variety of big data applications in domains like health care and scientific research, therefore more and more data owners are involved to outsource their data on cloud servers for great convenience in data management and mining. However, data sets like health records in electronic documents usually contain sensitive information, which brings about privacy concerns if the documents are released or shared to partially untrusted third-parties in cloud. A practical and widely used technique for data privacy preservation is to encrypt data before outsourcing to the cloud servers, which however reduces the data utility and makes many traditional data analytic operators like keyword-based top- $k$ k document retrieval obsolete. In this paper, we investigate the multi-keyword top- $k$ k search problem for big data encryption against privacy breaches, and attempt to identify an efficient and secure solution to this problem. Specifically, for the privacy concern of query data, we construct a special tree-based index structure and design a random traversal algorithm, which makes even the same query to produce different visiting paths on the index, and can also maintain the accuracy of queries unchanged under stronger privacy. For improving the query efficiency, we propose a group multi-keyword top- $k$ k search scheme based on the idea of partition, where a group of tree-based indexes are constructed for all documents. Finally, we combine these methods together into an efficient and secure approach to address our proposed top- $k$ k similarity search. Extensive experimental results on real-life data sets demonstrate that our proposed approach can significantly improve the capability of defending the privacy breaches, the scalability and the time efficiency of query processing over the state-of-the-art methods.

Aggregated multi-attribute query processing in edge computing for industrial IoT applications

The popularity of smart things constructs sensing networks for the Internet of Things (IoT), and promotes intelligent decision-makings to support industrial IoT applications, where multi-attribute query processing is an essential ingredient. Considering the huge number of smart things and large-scale of the network, traditional query processing mechanisms may not be applicable, since they mostly depend on a centralized index tree structure. To remedy this issue, this article proposes a multi-attribute aggregation query mechanism in the context of edge computing, where an energy-aware IR-tree is constructed to process query processing in single edge networks, while an edge node routing graph is established to facilitate query processing for marginal smart things contained in contiguous edge networks. This decentralized and localized strategy has shown its efficiency and applicability of query processing in IoT sensing networks. Experimental evaluation results demonstrate that this technique performs better than the rivals in reducing the traffic and energy consumption of the network.

Agent-Based System for Efficient kNN Query Processing with Comprehensive Privacy Protection

Recently, location based services (LBSs) have become increasingly popular due to advances in mobile devices and their positioning capabilities. In an LBS, the user sends a range of queries regarding his k-nearest neighbors (kNNs) that have common points of interests (POIs) based on his real geographic location. During the query sending, processing, and responding phases, private information may be collected by an attacker, either by tracking the real locations or by analyzing the sent queries. This compromises the privacy of the user and risks his/her safety in certain cases. Thus, the objective of this paper is to ensure comprehensive privacy protection, while also guaranteeing the efficiency of kNN query processing. Therefore, we propose an agent-based system for dealing with these issues. The system is managed by three software agents (selectorDL, fragmentorQ, and predictor). The selectorDL agent executes a Wise Dummy Selection Location (WDSL) algorithm to ensure the location privacy. The mission of the selectorDL agent is integrated with the mission of the fragmentorQ agent, which is to ensure the query privacy based on Left-Right Fragmentation (LRF) algorithm. To guarantee the efficiency of kNN processing, the predictor agent executes a prediction phase depending on a Cell Based Indexing (CBI) technique. Compared to similar privacy protection approaches, the proposed WDSL and LRF approaches showed higher resistance against location homogeneity attacks and query sampling attacks. In addition, the proposed CBI indexing technique obtains more accurate answers to kNN queries than the previous indexing techniques.