Query Size Research Articles

ArcMatch: high-performance subgraph matching for labeled graphs by exploiting edge domains

Consider a large labeled graph (network), denoted the target. Subgraph matching is the problem of finding all instances of a small subgraph, denoted the query, in the target graph. Unlike the majority of existing methods that are restricted to graphs with labels solely on vertices, our proposed approach, named can effectively handle graphs with labels on both vertices and edges. ntroduces an efficient new vertex/edge domain data structure filtering procedure to speed up subgraph queries. The procedure, called path-based reduction, filters initial domains by scanning them for paths up to a specified length that appear in the query graph. Additionally, ncorporates existing techniques like variable ordering and parent selection, as well as adapting the core search process, to take advantage of the information within edge domains. Experiments in real scenarios such as protein–protein interaction graphs, co-authorship networks, and email networks, show that s faster than state-of-the-art systems varying the number of distinct vertex labels over the whole target graph and query sizes.

Efficient computation of Top-k G-Skyline groups on large-scale database

The G-Skyline query aims to return the best groups that are not g-dominated by any other group of equal size; this approach plays an important role in many fields, such as multiobjective decision-making. The top-k G-Skyline query has important implications since the output size of the G-Skyline query is often too large to make decisions. Due to the massive number of candidate groups and the high index construction consumption, the existing top-k G-Skyline algorithms cannot handle large-scale databases well. In this paper, an efficient algorithm is proposed called the TGPE (Top-kG-Skyline algorithm based on Presorting and Enumeration) to rapidly compute the top-k G-Skyline groups for large-scale databases. TGPE proposes an efficient scanning method based on a presorted table to obtain tuples making up G-Skyline groups and their dominant relationships. TGPE does not need to repeatedly reconstruct the index for different skyline criteria. Three computational lemmas are presented based on sorting information and monotonicity, and they are utilized to reduce the number of candidate groups to obtain results rapidly. Finally, extensive experiments on synthetic and real datasets verify that TGPE has significant performance advantages over those of the existing algorithms.

Selectivity Estimation for Queries Containing Predicates over Set-Valued Attributes

Selectivity estimation aims to estimate the size of query results size accurately and efficiently. Despite being a well-researched area for decades, most existing estimators are designed to handle comparison predicates over numeric and categorical data. Nevertheless, Set-valued data are ubiquitous in many applications such as information retrieval and recommendation systems. However, these estimators may not be effective for handling predicates over set-valued data. In this work, we presents novel techniques for selectivity estimation on queries involving predicates over set-valued attributes. We first propose the set-valued column factorization problem, whereby each each set-valued column is converted to multiple numeric subcolumns, and set containment predicates are converted to numeric comparison predicates. This enables us to leverage any existing estimator to perform selectivity estimation. We then develop two methods for column factorization and query conversion, namely ST and STH. We integrate ST and STH into three estimators, Postgres, Neurocard, and DeepDB. We then conduct a comprehensive empirical analysis by comparing our approach against three baselines across three different datasets. The experimental results demonstrate that our methods exhibit superior estimation accuracy while maintaining high efficiency compared to the baseline techniques.

EnsemblQueryR: fast, flexible and high-throughput querying of Ensembl LD API endpoints in R.

We present ensemblQueryR, an Rpackage for querying Ensembl linkage disequilibrium (LD) endpoints. This package is flexible, fast and user-friendly, and optimised for high-throughput querying. ensemblQueryR uses functions that are intuitive and amenable to custom code integration, familiar R object types as inputs and outputs as well as providing parallelisation functionality. For each Ensembl LD endpoint, ensemblQueryR provides two functions, permitting both single- and multi-query modes of operation. The multi-query functions are optimised for large query sizes and provide optional parallelisation to leverage available computational resources and minimise processing time. We demonstrate improved computational performance of ensemblQueryR over an exisiting tool in terms of random access memory (RAM) usage and speed, delivering a 10-fold speed increase whilst using a third of the RAM. Finally, ensemblQueryR is near-agnostic to operating system and computational architecture through Docker and singularity images, making this tool widely accessible to the scientific community.

Blockchain-Assisted Secure Data Sharing Protocol With a Dynamic Multiuser Keyword Search in IIoT

The industrial internet of things (IIoT) and cloud computing have developed rapidly in recent years. Many enterprises are willing to outsource lightweight devices’ industrial data via the cloud to lower manufacturing costs and enhance production efficiency. The data sharers, however, usually have concerns about the security and privacy of their data stored in cloud outsourcing systems. Traditional certificateless searchable encryption primitives are tough to realize dynamically revocable and high-efficiency decryption. Furthermore, data sharing on untrusted devices may cause a single-key exposure problem. To address these issues, we propose a blockchain-assisted secure data-sharing protocol with a dynamic multi-user keyword search in IIoT (DMUKS), which utilizes blockchain-assisted techniques to solve a single-key exposure problem and to realize fast certificateless keyword search as well as dynamic user and key management. The proposed DMUKS scheme enjoys minor computation and communication overheads. It maintains a constant ciphertext and trapdoor query size as users increase and supports user addition and revocation. Moreover, it periodically supports key and ciphertext updating and is secure against keyword-guessing attacks under the random oracle model. The performance comparison and analysis demonstrate that it is more efficient and flexible than the existing data sharing with keyword search schemes.

Cardinality estimation of activity trajectory similarity queries using deep learning

Cardinality estimation, which involves estimating the result size of queries, is a critical aspect of query processing and optimization. Deep Neural Networks (DNNs) are data hungry, and being trained directly for cardinality estimation of activity trajectory similarity queries usually leads to poor performance. To address this problem, we propose two enhancements to improve accuracy and reduce the training data size: query slice and data slice. Query slice divides a query trajectory into three slices based on similarity dimensions (i.e., spatial, temporal, and textual). Data slice organizes similar trajectories into data slices. Furthermore, we design a global-local model to determine which local models should be used for a given query. We also extend above model to support similarity joins. Experimental results show that our method efficiently learns to estimate the cardinality, achieving superior accuracy and efficiency compared to state-of-the-art methods.

Profiling the BLAST bioinformatics application for load balancing on high-performance computing clusters

BackgroundThe Basic Local Alignment Search Tool (BLAST) is a suite of commonly used algorithms for identifying matches between biological sequences. The user supplies a database file and query file of sequences for BLAST to find identical sequences between the two. The typical millions of database and query sequences make BLAST computationally challenging but also well suited for parallelization on high-performance computing clusters. The efficacy of parallelization depends on the data partitioning, where the optimal data partitioning relies on an accurate performance model. In previous studies, a BLAST job was sped up by 27 times by partitioning the database and query among thousands of processor nodes. However, the optimality of the partitioning method was not studied. Unlike BLAST performance models proposed in the literature that usually have problem size and hardware configuration as the only variables, the execution time of a BLAST job is a function of database size, query size, and hardware capability. In this work, the nucleotide BLAST application BLASTN was profiled using three methods: shell-level profiling with the Unix “time” command, code-level profiling with the built-in “profiler” module, and system-level profiling with the Unix “gprof” program. The runtimes were measured for six node types, using six different database files and 15 query files, on a heterogeneous HPC cluster with 500+ nodes. The empirical measurement data were fitted with quadratic functions to develop performance models that were used to guide the data parallelization for BLASTN jobs.ResultsProfiling results showed that BLASTN contains more than 34,500 different functions, but a single function, RunMTBySplitDB, takes 99.12% of the total runtime. Among its 53 child functions, five core functions were identified to make up 92.12% of the overall BLASTN runtime. Based on the performance models, static load balancing algorithms can be applied to the BLASTN input data to minimize the runtime of the longest job on an HPC cluster. Four test cases being run on homogeneous and heterogeneous clusters were tested. Experiment results showed that the runtime can be reduced by 81% on a homogeneous cluster and by 20% on a heterogeneous cluster by re-distributing the workload.DiscussionOptimal data partitioning can improve BLASTN’s overall runtime 5.4-fold in comparison with dividing the database and query into the same number of fragments. The proposed methodology can be used in the other applications in the BLAST+ suite or any other application as long as source code is available.

A PID-Based kNN Query Processing Algorithm for Spatial Data.

As a popular spatial operation, the k-Nearest Neighbors (kNN) query is widely used in various spatial application systems. How to efficiently process a kNN query on spatial big data has always been an important research topic in the field of spatial data management. The centralized solutions are not suitable for spatial big data due to their poor scalability, while the existing distributed solutions are not efficient enough to meet the high real-time requirements of some spatial applications. Therefore, we introduce the Proportional Integral Derivative (PID) control technology into kNN query processing and propose a PID-based kNN query processing algorithm (PIDKNN) for spatial big data based on Spark. In this algorithm, the whole data space is divided into grid cells of the same size using the grid partition method, and the grid-based index is constructed. On this basis, the grid-based density peak clustering algorithm is used to cluster spatial data, and the corresponding PID parameters are set for each cluster. When performing kNN queries, the PID algorithm is used to estimate the radius growth step size of kNN queries, thereby realizing kNN query processing with a variable query radius growth step based on a feedback mechanism, which greatly improves the efficiency of kNN query processing. A series of experimental results show that the PIDKNN algorithm has good performance and scalability and is superior to the existing parallel kNN query processing methods.

Transformer for Nonintrusive Load Monitoring: Complexity Reduction and Transferability

Nonintrusive load monitoring (NILM) is to obtain individual appliance’s electricity consumption from aggregated smart meter data. In this article, we propose a middle window transformer model, termed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Midformer</i> , for NILM. Existing models are limited by high computational complexity, dependency on data, and poor transferability. In Midformer, we first exploit patchwise embedding to shorten the input length, and then reduce the size of queries in the attention layer by only using global attention on a few selected input locations at the center of the window to capture the global context. The cyclically shifted window technique is used to preserve connection across patches. We also follow the pretraining and fine-tuning paradigm to relieve the dependency on data, reduce the computation in modeling training, and enhance transferability of the model to unknown tasks and domains. Our experimental study using two real-world data sets demonstrates the superior performance and transferability of Midformer over three baseline models.

A plurality problem with three colors and query size three

The Plurality problem - introduced by Aigner - has many variants. In this article we deal with the following version: suppose we are given n balls, each of them colored by one of three colors. A plurality ball is one such that its color class is strictly larger than any other color class. Questioner asks a triplet (or a k-set in general), and Adversary as an answer gives the partition of the triplet (or the k-set) into color classes. Questioner wants to find a plurality ball as soon as possible or show that there is no such ball, while Adversary wants to postpone this.We denote by Ap(n,k) the largest number of queries needed to ask in the worst case if both play optimally. We provide an almost precise result in the case of even n by proving that for n≥4 even we have34n−2≤Ap(n,3)≤34n−12, and for n≥3 odd we have34n−O(log⁡n)≤Ap(n,3)≤34n−12.We also prove some bounds on the number of queries needed to ask in the case k≥3.

COORDINATE INDEXING ALGORITHMS TAKING INTO ACCOUNT THE CLASSIFICATION SIGNS-TERMS IN THE SUBJECT AREA

Libraries and Internet searchers are becoming increasingly frustrated with topical access results, in part because of the unmanageability of a wide range of search engines. The need to improve accuracy and limit the size of search queries was the main motivation for this study. The purpose of this article is to explore coordinate indexing algorithms taking into account the classification features-terms in the subject area. The study was written using the material modelling method (mathematical): pre-existing models were used, which offer partial coordination as fundamental to confirm their effectiveness in practice. The measurement method was used to analyse the numerical results. In the presented study, the features and characteristics of partial coordination were presented. In addition, the benefits of partial coordination have been studied. The methodology proposed by the authors focuses on the benefits of a “smart” query that can be executed in an OPAC environment without further effort on the part of the user; the user enters their keywords as before, but “smart” keyword coordination in documents avoids unsuitable partial matches. It was concluded that the order of ranking of documents on the subject should be included in the same way with those unverified records lacking partial coordination of partial documents.

A Hybrid Method for Equivalence Checking Between System Level and RTL

Deep State Sequence-based (DSS) equivalence checking and path-based equivalence checking have been successfully applied for verification of digital designs between System Level Model (SLM) and Register Transfer Level (RTL). The DSS-based equivalence checking method can validate designs without mapping information, but the query size for each DSS is large compared with path-based verification. The query size for path-based methods is small, but the number of comparisons is large. In this work, we combine the advantages of DSS-based methods and path-based methods. We use DSS-based methods to find the corresponding paths and use cut-points like in path-based methods to split the DSS to reduce the query size. Finite State Machine with Datapath (FSMD) is used to represent the SLM and RTL models. Experimental results demonstrate that our method can effectively validate the designs and reduce the query size for DSS-based equivalence checking method.

Multi-Server Weakly-Private Information Retrieval

Private information retrieval (PIR) protocols ensure that a user can download a file from a database without revealing any information on the identity of the requested file to the servers storing the database. While existing protocols strictly impose that no information is leaked on the file&#x2019;s identity, this work initiates the study of the tradeoffs that can be achieved by relaxing the perfect privacy requirement. We refer to such protocols as weakly-private information retrieval (WPIR) protocols. In particular, for the case of multiple noncolluding replicated servers, we study how the download rate, the upload cost, and the access complexity can be improved when relaxing the perfect privacy constraint. To quantify the information leakage on the requested file&#x2019;s identity we consider mutual information (MI), worst-case information leakage, and maximal leakage (MaxL). We present two WPIR schemes, denoted by Scheme A and Scheme B, based on two recent PIR protocols and show that the download rate of the former can be optimized by solving a convex optimization problem. We also show that Scheme A achieves an improved download rate compared to the recently proposed scheme by Samy <i>et al.</i> under the so-called <inline-formula> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula>-privacy metric. Additionally, a family of schemes based on partitioning is presented. Moreover, we provide an information-theoretic converse bound for the maximum possible download rate for the MI and MaxL privacy metrics under a practical restriction on the alphabet size of queries and answers. For two servers and two files, the bound is tight under the MaxL metric, which settles the WPIR capacity in this particular case. Finally, we compare the performance of the proposed schemes and their gap to the converse bound.

A Connection Access Mechanism of Distributed Network based on Block Chain

Cross-border payment optimization technology based on block chain has become a hot spot in the industry. The traditional method mainly includes the block feature detection method, the fuzzy access method, the adaptive scheduling method, which perform related feature extraction and quantitative regression analysis on the collected distributed network connection access data, and combine the fuzzy clustering method to optimize the data access design, and realize the group detection and identification of data in the block chain. However, the traditional method has a large computational overhead for distributed network connection access, and the packet detection capability is not good. This paper constructs a statistical sequence model of adaptive connection access data to extract the descriptive statistical features of the distributed network block chain adaptive connection access data similarity. The performance of the strategy retrieval efficiency in the experiment is tested based on the strategy management method. The experiment performs matching query tests on the test sets of different query sizes. The different parameters for error rate and search delay test are set to evaluate the impact of different parameters on retrieval performance. The calculation method of single delay is the total delay or the total number of matches. The optimization effect is mainly measured by the retrieval delay of the strategy in the strategy management contract; the smaller the delay, the higher the execution efficiency, and the better the retrieval optimization effect.

On non-adaptive majority problems of large query size

We are given $n$ balls and an unknown coloring of them with two colors. Our goal is to find a ball that belongs to the larger color class, or show that the color classes have the same size. We can ask sets of $k$ balls as queries, and the problem has different variants, according to what the answers to the queries can be. These questions has attracted several researchers, but the focus of most research was the adaptive version, where queries are decided sequentially, after learning the answer to the previous query. Here we study the non-adaptive version, where all the queries have to be asked at the same time.

SamQL: a structured query language and filtering tool for the SAM/BAM file format

BackgroundThe Sequence Alignment/Map Format Specification (SAM) is one of the most widely adopted file formats in bioinformatics and many researchers use it daily. Several tools, including most high-throughput sequencing read aligners, use it as their primary output and many more tools have been developed to process it. However, despite its flexibility, SAM encoded files can often be difficult to query and understand even for experienced bioinformaticians. As genomic data are rapidly growing, structured, and efficient queries on data that are encoded in SAM/BAM files are becoming increasingly important. Existing tools are very limited in their query capabilities or are not efficient. Critically, new tools that address these shortcomings, should not be able to support existing large datasets but should also do so without requiring massive data transformations and file infrastructure reorganizations.ResultsHere we introduce SamQL, an SQL-like query language for the SAM format with intuitive syntax that supports complex and efficient queries on top of SAM/BAM files and that can replace commonly used Bash one-liners employed by many bioinformaticians. SamQL has high expressive power with no upper limit on query size and when parallelized, outperforms other substantially less expressive software.ConclusionsSamQL is a complete query language that we envision as a step to a structured database engine for genomics. SamQL is written in Go, and is freely available as standalone program and as an open-source library under an MIT license, https://github.com/maragkakislab/samql/.

The parameterized complexity and kernelization of resilience for database queries

Given a database instance and a query on it whose result is initially non-empty, the resilience decision problem is to decide if there exist a small enough number of facts in the database instance such that the deletion of these facts empties the result of the given query. In this paper, we revisit the resilience decision problem. We investigate the parameterized complexity for various classes of database queries. We consider the factors including the query size and the number of variables, and present several intractable cases even from the perspective of parameterized complexity. Meanwhile, we refine the characteristics of resilience for self-join-free conjunctive queries containing triads, and show that it is still NP-hard even if the structure of the input database instance is simple. This result implies the hardness essentially comes from the parity of triangle sequence instead of the complicate (non-planar) intersections of cycles. On the other hand, we also obtain some positive results showing that the resilience decision problem is still fixed parameter tractable for an important case through kernelization. Our work demonstrates a new insight for employing resilience computation in database operations.

Finite Open-world Query Answering with Number Restrictions

Open-world query answering is the problem of deciding, given a set of facts, conjunction of constraints, and query, whether the facts and constraints imply the query. This amounts to reasoning over all instances that include the facts and satisfy the constraints. We study finite open-world query answering (FQA), which assumes that the underlying world is finite and thus only considers the finite completions of the instance. The major known decidable cases of FQA derive from the following: the guarded fragment of first-order logic, which can express referential constraints (data in one place points to data in another) but cannot express number restrictions such as functional dependencies; and the guarded fragment with number restrictions but on a signature of arity only two. In this article, we give the first decidability results for FQA that combine both referential constraints and number restrictions for arbitrary signatures: We show that, for unary inclusion dependencies and functional dependencies, the finiteness assumption of FQA can be lifted up to taking the finite implication closure of the dependencies. Our result relies on new techniques to construct finite universal models of such constraints for any bound on the maximal query size.

Virtualization Based Efficient Service Matching and Discovery in Internet of Things

Internet of Things (IoT) is getting more popular day by day, which triggers its adoption for solving domain specific problems. Cities are becoming smart by gathering the context knowledge through sensors and controlling specific parameters through actuators. Dynamically discovering and integrating different data streams from different sensors is a major challenge these days. In this paper, a service matchmaking algorithm is presented for service discovery utilizing IoT devices and services in a particular geographic area. It helps us to identify services based on a variety of parameters (location, query size and processing time, etc.). Customization of service selection and discovery are also explored. The conceptual framework is provided for the proposed model along with a matchmaking algorithm based on IoT devices virtualization. The simulation results elaborate the increased complexity of processing time with respect to the increasing pool of available services. The average processing time varies as the number of conditions are multiplied. Query size and complexity increases with additional number of filters and conditions which results in the reduction of the number of matching services. Moreover, upon decreasing the radius of geographic search area, the number of candidate services decreases for service matching algorithm. This is based on the assumption that IoT devices and services are evenly distributed in a given geographic area. Similarly, the remaining energy of IoT devices is also assumed to be uniformly distributed and, therefore, if we are interested in IoT devices or services with more residual energy, then a limited number of IoT devices or services will fulfill this criterion.

Optimal Algorithms for Ranked Enumeration of Answers to Full Conjunctive Queries.

We study ranked enumeration of join-query results according to very general orders defined by selective dioids. Our main contribution is a framework for ranked enumeration over a class of dynamic programming problems that generalizes seemingly different problems that had been studied in isolation. To this end, we extend classic algorithms that find the k-shortest paths in a weighted graph. For full conjunctive queries, including cyclic ones, our approach is optimal in terms of the time to return the top result and the delay between results. These optimality properties are derived for the widely used notion of data complexity, which treats query size as a constant. By performing a careful cost analysis, we are able to uncover a previously unknown trade-off between two incomparable enumeration approaches: one has lower complexity when the number of returned results is small, the other when the number is very large. We theoretically and empirically demonstrate the superiority of our techniques over batch algorithms, which produce the full result and then sort it. Our technique is not only faster for returning the first few results, but on some inputs beats the batch algorithm even when all results are produced.