Query Time Research Articles

Two-layer partitioned and deletable deep bloom filter for large-scale membership query

The recently proposed Learned Bloom Filter (LBF) provides a new perspective on large-scale membership queries by using machine learning to replace the traditional bloom filter. However, reducing the false positive rate (FPR) of the learned model with small memory usage, and supporting deletion efficiently become the new issues. In this paper, we propose a novel Two-layer Partitioned and Deletable Deep Bloom Filter (PDDBF) for large-scale membership query, which can reduce the FPR with small memory usage and support deletion efficiently. The proposed PDDBF consists of three main parts: (1) Data partition. To improve the classification accuracy of the learned model, the K-means cluster with the elbow method is used for the data partition. (2) Deep Bloom Filter. To reduce the FPR, deep learning models are used to construct multiple independent learning mechanisms, which correspond to the clusters obtained by part1. (3) Partitioned backup filter. To support deletion under the premise of ensuring low FPR and reducing query time consumption, combine the perfect hash (PH) table and counting bloom filters (CBFs) on the basis of the partition bloom filter. Experiments show that the proposed PDDBF reduces the FPR 87.13% with the same memory usage compared with the state-of-the-art PLBF on real-world URLs data set. Moreover, the PDDBF reduces the FPR 99.68% with the same memory usage and reduces the query time consumption to 2.61x that of the PLBF after data deletion, respectively.

Changes in web search query behavior of English-to-Chinese translation trainees

This article delves into the evolution of web search query behavior among translation trainees over a semester, using a repeated measures design. While prior research in Translation Studies has predominantly focused on information-seeking patterns and types of resources used, a crucial gap remains in understanding the web search query behavior of diverse translation practitioners. Specifically, this investigation analyzes intra-subject and inter-task level changes in web search query behavior within a group of 19 first-year master's students enrolled in an English-to-Chinese translation course, building upon key literature in the field. The statistical analysis conducted reveals changes in the examined variables—query time, complexity, and language—highlighting the importance of targeted research and training to enhance web search skills and search engine proficiency. These findings carry relevant implications for translation practitioners, particularly as new AI technologies continue to expand and enhance their search toolbox and skillset.

PMMTss: A Parallel Multi-Way Merging-Based Trajectory Similarity Search for a Million Metro Passengers

Trajectory similarity search (TSS) is a common operation for spatiotemporal data analysis. However, the existing TSS methods are mainly focused on GPS trajectories produced by moving objects such as vehicles. Further, these corresponding optimization strategies cannot be directly applied in the metro scenario because the metro passenger trajectory is totally different from the GPS trajectory. To fill this gap, we systematically analyze the unique spatiotemporal characteristics of metro passenger trajectories and propose a similarity search solution named PMMTss for the metro system. The core idea of this solution has two key points: first, we design a multi-layer index based on the spatiotemporal feature of metro trajectories, and all points of a trajectory are stored in this index. Second, we design a parallel multi-way merging-based trajectory similar search method, in which the candidate trajectory segments are merged and filtered. We evaluate this solution on a large dataset (Shenzhen Metro data for 3 consecutive months, 6.976 million trajectories with 260 million records). When lengths of input trajectories are 16, 32, and 64, respectively, the corresponding search times are 0.004 s, 0.016 s, and 0.036 s, respectively. Compared to the baseline PPJion+, the query times are reduced by 99.7%, 98.8%, and 97.6%, respectively.

Efficient Multi-Query Bi-Objective Search via Contraction Hierarchies

Contraction Hierarchies (CHs) have been successfully used as a preprocessing technique in single-objective graph search for finding shortest paths. However, only a few existing works on utilizing CHs for bi-objective search exist, and none of them uses CHs to compute Pareto frontiers. This paper proposes an CH-based approach capable of efficiently computing Pareto frontiers for bi-objective search along with several speedup techniques. Specifically, we propose a new preprocessing approach that computes CHs with fewer edges than the existing preprocessing approach, which reduces both the preprocessing times (up to 3x in our experiments) and the query times. Furthermore, we propose a partial-expansion technique, which dramatically speeds up the query times. We demonstrate the advantages of our approach on road networks with 1 to 14 million states. The longest preprocessing time is less than 6 hours, and the average speedup in query times is roughly two orders of magnitude compared to BOA*, a state-of-the-art single-query bi-objective search algorithm.

Sieve: A Learned Data-Skipping Index for Data Analytics

Modern data analytics services are coupled with external data storage services, making I/O from remote cloud storage one of the dominant costs for query processing. Techniques such as columnar block-based data organization and compression have become standard practices for these services to save storage and processing cost. However, the problem of effectively skipping irrelevant blocks at low overhead is still open. Existing data-skipping efforts maintain lightweight summaries (e.g., min/max, histograms) for each block to filter irrelevant data. However, such techniques ignore patterns in real-world data, enabling ineffective use of the storage budget and may cause serious false positives. This paper presents Sieve, a learning-enhanced index designed to efficiently filter out irrelevant blocks by capturing data patterns. Specifically, Sieve utilizes piece-wise linear functions to capture block distribution trends over the key space. Based on the captured trends, Sieve trades off storage consumption and false positives by grouping neighboring keys with similar block distributions into a single region. We have evaluated Sieve using Presto, and experiments on real-world datasets demonstrate that Sieve achieves up to 80% reduction in blocks accessed and 42% reduction in query times compared to its counterparts.

The analysis of Java ORM frameworks performance in terms of analytical data processing

The aim of this paper is to evaluate Java ORM frameworks in terms of analytical data processing. The analysis includes following technologies: Hibernate, Apache Cayenne, EclipseLink and DataNucleus. Article contains characteristics and importance of ORM technologies, as well as the research of related literature. The main study has been performed with the use of implemented Java applications that enabled to run and measure the execution time of analytical queries with various levels of complexity. The analysis of the obtained results enabled to define Hibernate as the most efficient technology for analytical data processing. Additionally the limitations of examined tools and the significant differences between them have been identified and presented.

Efficient and privacy-preserving multi-party skyline queries in online medical primary diagnosis

Wireless body area networks enable data collection from wearable devices, thereby allowing online medical primary diagnosis via cloud computing. Data security and diagnosis accuracy are two main concerns in the online medical primary diagnosis system. While traditional solutions can ensure the confidentiality of online data, their incapacity to integrate data from multiple users restricts the development of accurate diagnostic models and leads to low accuracy. Recently, a medical preliminary diagnosis scheme with improved accuracy was proposed, which employs skyline computation to construct a precise diagnostic model using multiple medical datasets. However, their scheme requires a trusted third party and experiences excessive query time. To address this issue, we present an Effective and Privacy-preserving Multi-party Skyline diagnosis scheme (EPMS) that offers even higher accuracy and extremely fast diagnosis without trusted third parties. Specifically, we devise several sub-protocols to support secure skyline computation. By integrating our protocols with privacy matrix techniques, the cloud server can generate a comprehensive diagnostic model from multiple data sources, offering accurate diagnosis services without disclosing any users’ personal information. We implement our scheme and conduct extensive experiments, which showed that our approach achieves a speedup of approximately 200× in query time and nearly 20% improvement in accuracy.

Fast Online Hashing with Multi-Label Projection

Hashing has been widely researched to solve the large-scale approximate nearest neighbor search problem owing to its time and storage superiority. In recent years, a number of online hashing methods have emerged, which can update the hash functions to adapt to the new stream data and realize dynamic retrieval. However, existing online hashing methods are required to update the whole database with the latest hash functions when a query arrives, which leads to low retrieval efficiency with the continuous increase of the stream data. On the other hand, these methods ignore the supervision relationship among the examples, especially in the multi-label case. In this paper, we propose a novel Fast Online Hashing (FOH) method which only updates the binary codes of a small part of the database. To be specific, we first build a query pool in which the nearest neighbors of each central point are recorded. When a new query arrives, only the binary codes of the corresponding potential neighbors are updated. In addition, we create a similarity matrix which takes the multi-label supervision information into account and bring in the multi-label projection loss to further preserve the similarity among the multi-label data. The experimental results on two common benchmarks show that the proposed FOH can achieve dramatic superiority on query time up to 6.28 seconds less than state-of-the-art baselines with competitive retrieval accuracy.

T-Rex: Optimizing Pattern Search on Time Series

Pattern search is an important class of queries for time series data. Time series patterns often match variable-length segments with a large search space, thereby posing a significant performance challenge. The existing pattern search systems, for example, SQL query engines supporting MATCH_RECOGNIZE, are ineffective in pruning the large search space of variable-length segments. In many cases, the issue is due to the use of a restrictive query language modeled on time series points and a computational model that limits search space pruning. We built T-ReX to address this problem using two main building blocks: first, a MATCH_RECOGNIZE language extension that exposes the notion of segment variable and adds new operators, lending itself to better optimization; second, an executor capable of pruning the search space of matches and minimizing total query time using an optimizer. We conducted experiments using 5 real-world datasets and 11 query templates, including those from existing works. T-ReX outperformed an optimized NFA-based pattern search executor by 6x in median query time and an optimized tree-based executor by 19X.

BtrBlocks: Efficient Columnar Compression for Data Lakes

Analytics is moving to the cloud and data is moving into data lakes. These reside on object storage services like S3 and enable seamless data sharing and system interoperability. To support this, many systems build on open storage formats like Apache Parquet. However, these formats are not optimized for remotely-accessed data lakes and today's high-throughput networks. Inefficient decompression makes scans CPU-bound and thus increases query time and cost. With this work we present BtrBlocks, an open columnar storage format designed for data lakes. BtrBlocks uses a set of lightweight encoding schemes, achieving fast and efficient decompression and high compression ratios.

Triangle Influence Supersets for Fast Distance Computation

AbstractWe present an acceleration structure to efficiently query the Signed Distance Field (SDF) of volumes represented by triangle meshes. The method is based on a discretization of space. In each node, we store the triangles defining the SDF behaviour in that region. Consequently, we reduce the cost of the nearest triangle search, prioritizing query performance, while avoiding approximations of the field. We propose a method to conservatively compute the set of triangles influencing each node. Given a node, each triangle defines a region of space such that all points inside it are closer to a point in the node than the triangle is. This property is used to build the SDF acceleration structure. We do not need to explicitly compute these regions, which is crucial to the performance of our approach. We prove the correctness of the proposed method and compare it to similar approaches, confirming that our method produces faster query times than other exact methods.

Engineering archive management model based on big data analysis and deep learning model

Abstract In the background of the era of big data, the information management system of engineering archives has become more comprehensive and perfect because of the application of information technology. The application of deep learning model makes the management of engineering archives more systematic, scientific and standardized, which greatly improves the quality and efficiency of engineering archives management. The progress of society and the development of the times have put forward higher requirements for digital storage technology. This paper combines the characteristics of the new technological era, analyses the characteristics of traditional information management in the context of data processing, artificial intelligence, deep learning and other data, proposes a method for developing and managing web archives based on Bootstrapping technology, introduces an information meta-evaluation mechanism to improve the quality of mining, and uses a long and short-term memory model to extract multi-type fine-grained archival information elements in the corpus. Finally, the Alex network was established to manage the archives in a categorised manner. The experimental results show that the query results of the proposed method for the target archives are 100% accurate, and the query time for individual archives is basically within 5s, which has good management effect.

Learned Data-aware Image Representations of Line Charts for Similarity Search

Finding line-chart images similar to a given line-chart image query is a common task in data exploration and image query systems, e.g. finding similar trends in stock markets or medical Electroencephalography images. The state-of-the-art approaches consider either data-level similarity (when the underlying data is present) or image-level similarity (when the underlying data is absent). In this paper, we study the scenario that during query time, only line-chart images are available. Our goal is to train a neural network that can turn these line-chart images into representations that are aware of the data used to generate these line charts, so as to learn better representations. Our key idea is that we can collect both data and line-chart images to learn such a neural network (at training step), while during query (or inference) time, we support the case that only line-chart images are provided. To this end, we present LineNet, a Vision Transformer-based Triplet Autoencoder model to learn data-aware image representations of line charts for similarity search. We design a novel pseudo labels selection mechanism to guide LineNet to capture both data-aware and image-level similarity of line charts. We further propose a diversified training samples selection strategy to optimize the learning process and improve the performance. We conduct both quantitative evaluation and case studies, showing that LineNet significantly outperforms the state-of-the-art methods for searching similar line-chart images.

ClipSim: A GPU-friendly Parallel Framework for Single-Source SimRank with Accuracy Guarantee

SimRank is an important metric to measure the topological similarity between two nodes in a graph. In particular, single-source and top-k SimRank has numerous applications in recommendation systems, network analysis, and web mining, etc. Mathematically, given a vertex, the computation of single-machine and single-source SimRank mainly lies in matrix-matrix operations. However, it is almost impossible to directly compute on large graphs. Thus, existing works yield to two main operations: a series of random walks, and sparse matrix and dense vector multiplication operations. This brings about high computation cost for SimRank on large graphs. In real-world applications, there is always the query time and accuracy trade-off, which hinders the computation of high-precision SimRank on large-scale graphs. To handle this problem, this paper proposesClipSim, the first GPU-friendly parallel framework that accelerates the single-source SimRank on GPU with accuracy guarantee. We design a novel data structure and GPU-friendly parallel algorithms for efficient computation of all the operations of SimRank on GPU. Moreover, our theoretical derivation enables ClipSim to largely reduce the number of random walks required for each node, while maintaining the same theoretical accuracy as the state-of-the-art algorithm, ExactSim. We conduct extensive experiments on real-world and synthetic datasets to demonstrate the accuracy and efficiency of ClipSim. The results show that compared with ExactSim, ClipSim obtains single-source SimRank vectors with the same accuracy and up to 160× faster computation time.

EAR-Oracle: On Efficient Indexing for Distance Queries between Arbitrary Points on Terrain Surface

Due to the advancement of geo-positioning technology, the terrain data has become increasingly popular and has drawn a lot of research effort from both academia and industry. The distance computation on the terrain surface is a fundamental and important problem that is widely applied in geographical information systems and 3D modeling. As could be observed from the existing studies, online computation of the distance on the terrain surface is very expensive. All existing index-based methods are only efficient under the case where the distance query must be performed among a small set of predefined points-of-interest known apriori. But, in general cases, they could not scale up to sizable datasets due to their intolerable oracle building time and space consumption. In this paper, we studied the arbitrary point-to-arbitrary point distance query on the terrain surface in which no assumption is imposed on the query points, and the distance query could be performed between any two arbitrary points. We propose an indexing structure, namely Efficient Arbitrary Point-to-Arbitrary Point Distance Oracle (EAR-Oracle), with theoretical guarantee on the accuracy, oracle building time, oracle size and query time. Our experiments demonstrate that our oracle enjoys excellent scalability and it scales up to enormous terrain surfaces but none of the existing index-based methods could be able to. Besides, it significantly outperforms all existing online computation methods by orders of magnitude in terms of the query time.

A Universal Question-Answering Platform for Knowledge Graphs

Knowledge from diverse application domains is organized as knowledge graphs (KGs) that are stored in RDF engines accessible in the web via SPARQL endpoints. Expressing a well-formed SPARQL query requires information about the graph structure and the exact URIs of its components, which is impractical for the average user. Question answering (QA) systems assist by translating natural language questions to SPARQL. Existing QA systems are typically based on application-specific human-curated rules, or require prior information, expensive pre-processing and model adaptation for each targeted KG. Therefore, they are hard to generalize to a broad set of applications and KGs. In this paper, we propose KGQAn, a universal QA system that does not need to be tailored to each target KG. Instead of curated rules, KGQAn introduces a novel formalization of question understanding as a text generation problem to convert a question into an intermediate abstract representation via a neural sequence-to-sequence model. We also develop a just-in-time linker that maps at query time the abstract representation to a SPARQL query for a specific KG, using only the publicly accessible APIs and the existing indices of the RDF store, without requiring any pre-processing. Our experiments with several real KGs demonstrate that KGQAn is easily deployed and outperforms by a large margin the state-of-the-art in terms of quality of answers and processing time, especially for arbitrary KGs, unseen during the training.

Black-Box Evasion Attack Method Based on Confidence Score of Benign Samples

Recently, malware detection models based on deep learning have gradually replaced manual analysis as the first line of defense for anti-malware systems. However, it has been shown that these models are vulnerable to a specific class of inputs called adversarial examples. It is possible to evade the detection model by adding some carefully crafted tiny perturbations to the malicious samples without changing the sample functions. Most of the adversarial example generation methods ignore the information contained in the detection results of benign samples from detection models. Our method extracts sequence fragments called benign payload from benign samples based on detection results and uses an RNN generative model to learn benign features embedded in these sequences. Then, we use the end of the original malicious sample as input to generate an adversarial perturbation that reduces the malicious probability of the sample and append it to the end of the sample to generate an adversarial sample. According to different adversarial scenarios, we propose two different generation strategies, which are the one-time generation method and the iterative generation method. Under different query times and append scale constraints, the maximum evasion success rate can reach 90.8%.

On approximate near-neighbors search under the (continuous) Fréchet distance in higher dimensions

Previous studies on Approximate Near-Neighbors Search (ANNS) among curves are either focused on curves in R1 or under the discrete Fréchet distance. In this paper, we propose the first data structure for curves under the (continuous) Fréchet distance in higher dimensions. Given a set P of n curves each with number of vertices at most m in Rd, and a fixed δ>0, we aim to preprocess P into a data structure so that for any given query curve Q with k vertices, we can efficiently report all curves in P whose Fréchet distances to Q are at most δ. In the case that k is given in the preprocessing stage, for any ε>0 we propose a deterministic data structure whose space is n⋅O(max⁡{(dε)kd,(Ddε2)kd}) that can answer (1+ε)δ-ANNS queries in O(kd) query time, where D is the diameter of P. Considering k as part of the query slightly changes the space to n⋅O(1/ε)md with O(kd) query time within an approximation factor of 5+ε. Moreover, we show that our generic data structure for ANNS can give an alternative treatment of the approximate subtrajectory range searching problem studied by de Berg et al. [1].

Fimpera: drastic improvement of Approximate Membership Query data-structures with counts.

High throughput sequencing technologies generate massive amounts of biological sequence datasets as costs fall. One of the current algorithmic challenges for exploiting these data on a global scale consists in providing efficient query engines on these petabyte-scale datasets. Most methods indexing those datasets rely on indexing words of fixed length k, called k-mers. Many applications, such as metagenomics, require the abundance of indexed k-mers as well as their simple presence or absence, but no method scales up to petabyte-scaled datasets. This deficiency is primarily because storing abundance requires explicit storage of the k-mers in order to associate them with their counts. Using counting Approximate Membership Queries (cAMQ) data structures, such as counting Bloom filters, provides a way to index large amounts of k-mers with their abundance, but at the expense of a sensible false positive rate. We propose a novel algorithm, called fimpera, that enables the improvement of any cAMQ performance. Applied to counting Bloom filters, our proposed algorithm reduces the false positive rate by two orders of magnitude and it improves the precision of the reported abundances. Alternatively, fimpera allows for the reduction of the size of a counting Bloom filter by two orders of magnitude while maintaining the same precision. fimpera does not introduce any memory overhead and may even reduces the query time. https://github.com/lrobidou/fimpera. Supplementary data are available at Bioinformatics online.

Fully Dynamic Connectivity in $O(\log n(\log\log n)^2)$ Amortized Expected Time

Dynamic connectivity is one of the most fundamental problems in dynamic graph algorithms. We present a randomized Las Vegas dynamic connectivity data structure with $O(\log n(\log\log n)^2)$ amortized expected update time and $O(\log n/\log\log\log n)$ worst case query time, which comes very close to the cell probe lower bounds of Patrascu and Demaine (2006) and Patrascu and Thorup (2011).