Query Algorithm Research Articles

Blockchain-Assisted Keyword Search Scheme for SWIM Service Based on Improved CSC-Cuckoo Filter

With the rapid development of the civil aviation industry, the demand for secure sharing of civil aviation data is increasing. As a global platform for secure sharing of civil aviation data, the security of the System Wide Information Management (SWIM) system is of great concern. SWIM adopts a service-oriented architecture and realizes the interaction of civil aviation data through a publish-subscribe model. To protect the service privacy while enabling subscribers to subscribe to the service quickly and securely, this paper proposes an efficient Circular Shift and Coalesce-Cuckoo Filter (CSC-CF)-based service Keyword Search (CCKS) scheme assisted by blockchain. This scheme uses the Symmetric-key Hidden Vector Encryption (SHVE) algorithm to encrypt and match service indexes and trapdoors to protect service privacy; the CSC-CF structure is applied to keyword search, which can effectively improve the efficiency of SWIM user subscription service. To improve search accuracy and space utilization, we propose the improved CCKS (ICCKS) scheme by optimizing the query algorithm of CSC-CF and setting a threshold for the number of keyword query failures. ICCKS improves search accuracy by 5–20% compared to CCKS when the filter’s space utilization is between 60 and 90%. Additionally, the scheme stores service topic indexes and registry on the blockchain and implements a smart contract to match indexes and subscription trapdoors, ensuring the integrity and trustworthiness of service topics and registry. Security analysis and experimental simulations demonstrate that the scheme is effective and secure in the SWIM system.

Semantic community query in a large‐scale attributed graph based on an attribute cohesiveness optimization strategy

AbstractThe task of a semantic community query is to obtain a subgraph based on a given query vertex (or vertex set) and other query parameters in an attributed graph such that belongs to , contains and satisfies a predefined community cohesiveness model. In most cases, existing community query models based on the network structure for traditional attributed networks usually lack community semantics. However, the features of vertex attributes, especially the attributes of the query vertices, which are closely related to the community semantics, are rarely considered in an attributed graph. Existing community query algorithms based on both structure cohesiveness and attribute cohesiveness usually do not take the attributes of the query vertex as an important factor of the community cohesiveness model, which leads to weak semantics of the communities. This paper proposes a semantic community query method named in a large‐scale attributed graph. First, the k‐core structure model is adopted as the structure cohesiveness of our community query model to obtain a subgraph of the original graph. Second, we define attribute cohesiveness based on the average distance between the query vertices and other vertices in terms of attributes in the community to prune the subgraph and obtain the semantic community. In order to improve the community query efficiency in large‐scale attributed graphs, applies two heuristic pruning strategies. The experimental results show that our method outperforms the existing community query methods in multiple evaluation metrics and is ideal for querying semantic communities in large‐scale attributed graphs.

Efficient Algorithms for Range Mode Queries in the Big Data Era

The mode is a fundamental descriptive statistic in data analysis, signifying the most frequent element within a dataset. The range mode query (RMQ) problem expands upon this concept by preprocessing an array A containing n natural numbers. This allows for the swift determination of the mode within any subarray A[a..b], thus optimizing the computation of the mode for a multitude of range queries. The efficacy of this process bears considerable importance in data analytics and retrieval across diverse platforms, including but not limited to online shopping experiences and financial auditing systems. This study is dedicated to exploring and benchmarking different algorithms and data structures designed to tackle the RMQ problem. The goal is to not only address the theoretical aspects of RMQ but also to provide practical solutions that can be applied in real-world scenarios, such as the optimization of an online shopping platform’s understanding of customer preferences, enhancing the efficiency and effectiveness of data retrieval in large datasets.

Geospatial indexing for sea–land navigation based on machine learning

The exponential growth of geospatial data has created a need for efficient data processing and indexing, especially for acquiring the required data around while traveling on land and sea. Machine learning based learned index has brought a new phase in the field of spatial data indexing on land and sea for fast and accurate acquisition of points of interest. Recently proposed spatial learned indexes and traditional spatial indexes are difficult to maintain their performance while traveling on land and sea due to high memory cost and time consumption. Considering the above issue, we propose SLIP, a novel in-memory spatial learned index. SLIP uses an optimized mapping function and a piecewise linear model to handle data updates (e.g., insertion and deletion) efficiently. Based on SLIP, we design a range query algorithm and a faster KNN(K-NearestNeighbor) query algorithm using the optimized radius expansion model to get the results more quickly. Experiments on real geospatial datasets and synthetic datasets used the electronic navigation chart software show that SLIP achieved an average improvement of 10x in query performance over traditional spatial indexes and state-of-the-art learned indexes. Therefore, SLIP provides an innovative idea for high-efficiency geospatial data processing and indexing, and offers an efficient method for acquiring the surrounding geographic data for future navigation on land and sea.

Efficient Parallel Processing of R-Tree on GPUs

R-tree is an important multi-dimensional data structure widely employed in many applications for storing and querying spatial data. As GPUs emerge as powerful computing hardware platforms, a GPU-based parallel R-tree becomes the key to efficiently port R-tree-related applications to GPUs. However, traditional tree-based data structures can hardly be directly ported to GPUs, and it is also a great challenge to develop highly efficient parallel tree-based data structures on GPUs. The difficulty mostly lies in the design of tree-based data structures and related operations in the context of many-core architecture that can facilitate parallel processing. We summarize our contributions as follows: (i) design a GPU-friendly data structure to store spatial data; (ii) present two parallel R-tree construction algorithms and one parallel R-tree query algorithm that can take the hardware characteristics of GPUs into consideration; and (iii) port the vector map overlay system from CPU to GPU to demonstrate the feasibility of parallel R-tree. Experimental results show that our parallel R-tree on GPU is efficient and practical. Compared with the traditional CPU-based sequential vector map overlay system, our vector map overlay system based on parallel R-tree can achieve nearly 10-fold speedup.

A privacy-preserving image retrieval scheme with access control based on searchable encryption in media cloud

With the popularity of the media cloud computing industry, individuals and organizations outsource image computation and storage to the media cloud server to reduce the storage burden. Media images usually contain a large amount of private information. To prevent disclosure of privacy of the image owners, media images are encrypted before uploading to the server. However, this operation will greatly limit the utilization of the image for the user, such as content-based image retrieval. We propose an efficient similarity query algorithm with access control based on Bkd-tree in this paper, in which a searchable encryption scheme is designed for similarity image retrieval, and the encrypted image is used to extract image features by a pre-trained CNN model. The Bkd-tree is utilized to generate an index tree for the image features to speed up retrieval and make it faster than linear indexing. Finally, the security performances of the proposed scheme is analyzed and the performance of this scheme is evaluated by experiments. The results show that the security of the image content and image features can be ensured, and it has a shorter retrieval time and higher retrieval efficiency.

Efficient algorithms for reachability and path queries on temporal bipartite graphs

Efficient algorithms for reachability and path queries on temporal bipartite graphs

Skyline query under multidimensional incomplete data based on classification tree

A method for skyline query of multidimensional incomplete data based on a classification tree has been proposed to address the problem of a large amount of useless data in existing skyline queries with multidimensional incomplete data, which leads to low query efficiency and algorithm performance. This method consists of two main parts. The first part is the proposed incomplete data weighted classification tree algorithm. In the first part, an incomplete data weighted classification tree is proposed, and the incomplete data set is classified using this tree. The data classified in the first part serves as the basis for the second step of the query. The second part proposes a skyline query algorithm for multidimensional incomplete data. The concept of optimal virtual points has been recently introduced, effectively reducing the number of comparisons of a large amount of data, thereby improving the query efficiency for incomplete data. Theoretical research and experimental analysis have shown that the proposed method can perform skyline queries for multidimensional incomplete data well, with high query efficiency and accuracy of the algorithm.

Research on Dynamic Updating Technology of Digital Maps for Mobile Communications Based on Multi source Remote Sensing Images

Abstract. With the rapid development of 5G networks in China, mobile communication network simulation technology has gradually become a widely studied direction in the academic and application fields. This study focuses on three-dimensional digital geographic information data for mobile communication, including a large amount of geographic information data that significantly affects the propagation of mobile communication radio waves, such as terrain height and ground use types. These data play a crucial role in coverage prediction, interference analysis, and frequency planning in various telecommunications planning software. This article starts from the perspective of mobile communication network management, and combines typical large-scale terrain data production practice projects for mobile communication to deeply study the process and system function design of combining geographic information systems with multi-source remote sensing data in mobile communication network monitoring and management. On this basis, this study designed and implemented a dynamic updated terrain database for mobile communication. This database can update geographic information data in real-time, ensuring the timeliness and accuracy of the data. At the same time, by optimizing the database structure and query algorithms, the speed and efficiency of data processing have been improved, providing efficient and reliable solutions for coverage prediction, optimization analysis, and planning management of large-scale mobile communication networks.In summary, dynamic updating database of three-dimensional digital geographic information has important application value in mobile communication network simulation technology. Through in-depth research and application of this database, the performance and service quality of mobile communication networks can be effectively improved, laying a solid foundation for future communication technology development.

Fast Matrix Multiplication for Query Processing

This paper studies how to use fast matrix multiplication to speed up query processing. As observed, computing a two-table join and then projecting away the join attribute is essentially the Boolean matrix multiplication problem, which can be significantly improved with fast matrix multiplication. Moving beyond this basic two-table query, we introduce output-sensitive algorithms for general join-project queries using fast matrix multiplication. These algorithms have achieved a polynomially large improvement over the classic Yannakakis framework. To the best of our knowledge, this is the first theoretical improvement for general acyclic join-project queries since 1981.

The Moments Method for Approximate Data Cube Queries

We investigate an approximation algorithm for various aggregate queries on partially materialized data cubes. Data cubes are interpreted as probability distributions, and cuboids from a partial materialization populate the terms of a series expansion of the target query distribution. Unknown terms in the expansion are just assumed to be 0 in order to recover an approximate query result. We identify this method as a variant of related approaches from other fields of science, that is, the Bahadur representation and, more generally, (biased) Fourier expansions of Boolean functions. Existing literature indicates a rich but intricate theoretical landscape. Focusing on the data cube application, we start by investigating worst-case error bounds. We build upon prior work to obtain provably optimal materialization strategies with respect to query workloads. In addition, we propose a new heuristic method governing materialization decisions. Finally, we show that well-approximated queries are guaranteed to have well-approximated roll-ups.

On the Two-sided Permutation Inversion Problem

In the permutation inversion problem, the task is to find the preimage of some challenge value, given oracle access to the permutation. This fundamental problem in query complexity appears in many contexts, particularly cryptography. In this work, we examine the setting in which the oracle allows for quantum queries to both the forward and the inverse direction of the permutation—except that the challenge value cannot be submitted to the latter. Within that setting, we consider three options for the inversion algorithm: whether it can get quantum advice about the permutation, whether the query algorithm can restrict the distribution with which the challenge input is sampled, and whether it must produce the entire preimage (search) or only the first bit (decision). We prove several theorems connecting the hardness of the resulting variations of the permutation inversion problem and establish lower bounds for them. Our results show that, perhaps surprisingly, the permutation inversion problem does not become significantly easier when the adversary is granted oracle access to the inverse—provided it cannot query the challenge itself.

A blockchain-based data storage architecture for Internet of Vehicles: Delay-aware consensus and data query algorithms

The recent development of the Internet of Vehicles (IoV) relies heavily on the secure store and analysis of reliable driving data. Blockchain technology has been widely used in the IoV field due to its security advantages, where consensus and query algorithms are two key modules determining its performance. However, existing solutions do not perform well enough in terms of delay performance, thus limiting their ability to support IoV environments with ultra-low-latency requirements. To address this challenge, we design a blockchain-based data storage architecture, focusing on the design of delay-aware consensus and data query algorithms. Specifically, to reduce the consensus delay, we propose a reputation-assisted and grouping-simplified Practical Byzantine Fault Tolerant (RGS-PBFT) algorithm. First, we develop a novel node credit value evaluation model by combining the behavior and configuration reputation to select the primary node with both high security and high computing capacity. Second, after determining the primary node, we reduce the communication complexity of traditional PBFT from O(Z2) to O(Z43) by establishing a grouping-simplified consensus structure. These contributions effectively reduce the consensus delay. To reduce the query delay and index construction time, we propose an index query algorithm based on the data tracking chain (DTC-Index query). By introducing the LevelDB database, it establishes a transaction index chain coupled to vehicle ID in the blockchain. This approach avoids the time-consuming block traversal during data querying. Experimental results demonstrate that the proposed algorithms outperform existing solutions, achieving significant improvements in both consensus delay and query time reduction. Notably, our solutions offer exceptional delay performance, making them well-suited for IoV environments.

Rapid multiple protein sequence search by parallel and heterogeneous computation.

Protein sequence database search and multiple sequence alignment generation is a fundamental task in many bioinformatics analyses. As the data volume of sequences continues to grow rapidly, there is an increasing need for efficient and scalable multiple sequence query algorithms for super-large databases without expensive time and computational costs. We introduce Chorus, a novel protein sequence query system that leverages parallel model and heterogeneous computation architecture to enable users to query thousands of protein sequences concurrently against large protein databases on a desktop workstation. Chorus achieves over 100× speedup over BLASTP without sacrificing sensitivity. We demonstrate the utility of Chorus through a case study of analyzing a ∼1.5-TB large-scale metagenomic datasets for novel CRISPR-Cas protein discovery within 30 min. Chorus is open-source and its code repository is available at https://github.com/Bio-Acc/Chorus.

IB-CBB: an improved spatial index considering intersection based on clipped bounding boxes

ABSTRACT Efficiently querying multiple spatial datasets is a challenging task in geoscience. The majority of spatial processing techniques use minimum bounding box (MBB) to approximate neighbouring spatial objects and and place them adjacent in the spatial index. However, due to the existence of redundant space in MBB of these methods, this problem can significantly reduce the query efficiency. In this paper, we propose a novel two-stage adaptive method of clipping the bounding box in spatial query, called IB-CBB (Intersection Based Clipped Bounding Boxes). The first stage employs a clipped bounding box, which records the redundant spatial spaces within the bounding box of the spatial index by calculating the clip points. As a result, the computational complexity of indexed child nodes in the query process is reduced. The second stage optimizes the above query algorithm by judging the intersection of the query box and the MBB of index node, significantly reducing the query time. Experiments demonstrate that IB-CBB outperforms the baseline method in terms of reducing the computational time.

Variational learning algorithms for quantum query complexity

Quantum query complexity is pivotal in the analysis of quantum algorithms, encompassing well-known examples like search and period-finding algorithms. These algorithms typically involve a sequence of unitary operations and oracle calls dependent on an input variable. In this study, we introduce a variational learning approach to explore quantum query complexity. Our method employs an efficient parameterization of the unitary operations and utilizes a loss function derived from the algorithm’s error probability. We apply this technique to various quantum query complexities, notably devising a new algorithm that resolves the 5-bit Hamming modulo problem with four queries, addressing an open question from Cornelissen et al (2021 arXiv:2112.14682). This finding is corroborated by a semidefinite programming (SDP) approach. Our numerical method exhibits superior memory efficiency compared to SDP and can identify quantum query algorithms (QQAs) that require a smaller workspace register dimension, an aspect not optimized by SDP. These advancements present a significant step forward in the practical application and understanding of QQAs.

Data Management for ML-Based Analytics and Beyond

The increasing capabilities of machine learning (ML) has enabled the deployment of ML methods in a variety of applications, ranging from unstructured data analytics to autonomous vehicles. Due to the volumes of data over which ML is deployed, it is infeasible for humans to monitor deployments: the Tesla fleet of vehicles produces exabytes of data and millions of hours of video per day. As a result, ML deployments can fail in unexpected and catastrophic ways. In this work, we highlight three important but underlooked aspects of ML deployment pipelines: (1) managing high-quality training data, (2) monitoring ML errors at deployment time, and (3) connecting end use to deployment algorithms. We first demonstrate that training labels are often erroneous, contrary to standard practice, even when labeled by leading vendors. We then demonstrate that standard methods of deploying ML methods can lead to downstream errors. As a first step toward addressing these issues, we review and contextualize two abstractions for finding errors in training data and deployments. We further describe how to improve algorithms for analytics queries as a case study for optimizing ML pipelines end to end.

Research on Informatization Construction of Educational Management Innovation Path in Colleges and Universities in the Intelligent Era

Abstract The prevailing education management systems in colleges and universities are antiquated, adversely impacting not only the administrative efficiency but also the formulation of faculty teaching plans and class schedules, as well as the management of student electives and credit accrual. This paper presents an innovation and optimization of the existing system, wherein education management processes are categorized into seven distinct modules. It delineates the flow and integration of heterogeneous data sources within the system, leveraging heterogeneous database technology as the foundational framework. Subsequently, we redesign heterogeneous computation scheduling and adaptation, enhancing distributed database allocation and query algorithms. The culmination of this research is the development of a novel pathway for the education management information system in higher education institutions. Upon empirical testing of the system proposed in this paper, we explore its practical implications through IPA analysis. User satisfaction is notably high in areas such as A4 invigilation information, A13 subjective grading of routine assessments, A14 querying and feedback on grades, and A15 remediation of failed courses, with performance ratings of 4.77008, 4.73981, 4.8961, and 4.86558 respectively. These indices predominantly reside in the “continue to maintain” and “oversupply” zones, indicating the substantial efficacy of the innovative college management information system introduced herein. This study significantly contributes to the standardization and intellectualization of college administrative processes, proposing new paradigms and methodologies for the integration of educational management and information technology within tertiary institutions.

Scaling Package Queries to a Billion Tuples via Hierarchical Partitioning and Customized Optimization

A package query returns a package---a multiset of tuples---that maximizes or minimizes a linear objective function subject to linear constraints, thereby enabling in-database decision support. Prior work has established the equivalence of package queries to Integer Linear Programs (ILPs) and developed the SketchRefine algorithm for package query processing. While this algorithm was an important first step toward supporting prescriptive analytics scalably inside a relational database, it struggles when the data size grows beyond a few hundred million tuples or when the constraints become very tight. In this paper, we present Progressive Shading, a novel algorithm for processing package queries that can scale efficiently to billions of tuples and gracefully handle tight constraints. Progressive Shading solves a sequence of optimization problems over a hierarchy of relations, each resulting from an ever-finer partitioning of the original tuples into homogeneous groups until the original relation is obtained. This strategy avoids the premature discarding of high-quality tuples that can occur with SketchRefine. Our novel partitioning scheme, Dynamic Low Variance, can handle very large relations with multiple attributes and can dynamically adapt to both concentrated and spread-out sets of attribute values, provably outperforming traditional partitioning schemes such as kd-tree. We further optimize our system by replacing our off-the-shelf optimization software with customized ILP and LP solvers, called Dual Reducer and Parallel Dual Simplex respectively, that are highly accurate and orders of magnitude faster.

Toward Answering Federated Spatial Range Queries Under Local Differential Privacy

Federated analytics (FA) over spatial data with local differential privacy (LDP) has attracted considerable research attention recently. Existing solutions for this problem mostly employ a uniform grid (UG) structure, which recursively decomposes the whole spatial domain into fine‐grained regions in the distributed setting. In each round, the sampled clients perturb their locations using a random response mechanism with a fixed probability. This approach, however, cannot encode the client’s location effectively and will lead to ill‐suited query results. To address the deficiency of existing solutions, we propose LDP‐FSRQ, a spatial range query algorithm that relies on a hybrid spatial structure composed of the UG and quad‐tree with nonuniform perturbation (NUP) probability to encode and perturb clients’ locations. In each iteration of LDP‐FSRQ, each client adopts the quad‐tree to encode his/her location into a binary string and uses four local perturbation mechanisms to protect the encoded string. Then, the collector prunes the quad‐tree of the current round according to the clients’ reports and shares the pruned tree with the clients of the next round. We demonstrate the application of LDP‐FSRQ on Beijing, Landmark, Check‐in, and NYC datasets, and the experimental results show that our approach outperforms its competitors in terms of queries’ utility.