Filter Data Structure Research Articles

Enhancing Iris Authentication for Managing Multiple Cancellations: Leveraging Quotient Filters

Biometric authentication methods have become increasingly popular for their ability to offer secure and convenient access control. However, concerns about the privacy and security of biometric data have arisen. In this study, we present a novel approach to address these concerns by proposing a cancellable biometric encryption technique for secure and format-preserving iris authentication. Our method leverages the Quotient Filter data structure to transform encrypted iris templates into cancellable templates while preserving their original format. We carefully select an appropriate format-preserving encryption algorithm for iris templates and design a mapping scheme to achieve cancellability. To assess the effectiveness and performance of our approach, extensive experiments are conducted. The quantitative results indicate the efficiency and efficacy of our cancellable biometric encryption technique using the Quotient Filter. Our innovation, the Iris Authentication for Multiple Cancelled Instances Using a Quotient Filter (IAMCIQF), demonstrates competitive performance across several key metrics. IAMCIQF achieves a high level of security strength and strikes a balance between security and efficiency in terms of key size, encryption and decryption speeds and storage efficiency when compared to other existing techniques. The quantitative outcomes underscore IAMCIQF’s potential as a promising solution for attaining secure and format-preserving iris authentication, addressing critical concerns about biometric data security.

Fast and space-efficient taxonomic classification of long reads with hierarchical interleaved XOR filters.

Metagenomic long-read sequencing is gaining popularity for various applications, including pathogen detection and microbiome studies. To analyze the large data created in those studies, software tools need to taxonomically classify the sequenced molecules and estimate the relative abundances of organisms in the sequenced sample. Because of the exponential growth of reference genome databases, the current taxonomic classification methods have large computational requirements. This issue motivated us to develop a new data structure for fast and memory-efficient querying of long reads. Here, we present Taxor as a new tool for long-read metagenomic classification using a hierarchical interleaved XOR filter data structure for indexing and querying large reference genome sets. Taxor implements several k-mer-based approaches, such as syncmers, for pseudoalignment to classify reads and an expectation-maximization algorithm for metagenomic profiling. Our results show that Taxor outperforms state-of-the-art tools regarding precision while having a similar recall for long-read taxonomic classification. Most notably, Taxor reduces the memory requirements and index size by >50% and is among the fastest tools regarding query times. This enables real-time metagenomics analysis with large reference databases on a small laptop in the field.

CRBF: Cross-Referencing Bloom-Filter-Based Data Integrity Verification Framework for Object-Based Big Data Transfer Systems

Various components are involved in the end-to-end path of data transfer. Protecting data integrity from failures in these intermediate components is a key feature of big data transfer tools. Although most of these components provide some degree of data integrity, they are either too expensive or inefficient in recovering corrupted data. This problem highlights the need for application-level end-to-end integrity verification during data transfer. However, the computational, memory, and storage overhead of big data transfer tools can be a significant bottleneck for ensuring data integrity due to the large size of the data. This paper proposes a novel framework for data integrity verification in big data transfer systems using a cross-referencing Bloom filter. This framework has three advantages over state-of-the-art data integrity techniques: lower computation and memory overhead and zero false-positive errors for a limited number of elements. This study evaluates the computation, memory, recovery time, and false-positive overhead for the proposed framework and compares them with state-of-the-art solutions. The evaluation results indicate that the proposed framework is efficient in detecting and recovering from integrity errors while eliminating false positives in the Bloom filter data structure. In addition, we observe negligible computation, memory, and recovery overheads for all workloads.

IMF-PR: An Improved Morton-Filter-Based Pseudonym-Revocation Scheme in VANETs.

Vehicle ad hoc networks (VANETs) are special wireless networks which help vehicles to obtain continuous and stable communication. Pseudonym revocation, as a vital security mechanism, is able to protect legal vehicles in VANETs. However, existing pseudonym-revocation schemes suffer from the issues of low certificate revocation list (CRL) generation and update efficiency, along with high CRL storage and transmission costs. In order to solve the above issues, this paper proposes an improved Morton-filter-based pseudonym-revocation scheme for VANETs (IMF-PR). IMF-PR establishes a new distributed CRL management mechanism to maintain a low CRL distribution transmission delay. In addition, IMF-PR improves the Morton filter to optimize the CRL management mechanism so as to improve CRL generation and update efficiency and reduce the CRL storage overhead. Moreover, CRLs in IMF-PR store illegal vehicle information based on an improved Morton filter data structure to improve the compress ratio and the query efficiency. Performance analysis and simulation experiments showed that IMF-PR can effectively reduce storage by increasing the compression gain and reducing transmission delay. In addition, IMF-PR can also greatly improve the lookup and update throughput on CRLs.

Research on complex attribute big data classification based on iterative fuzzy clustering algorithm

In order to overcome the low classification accuracy of traditional methods, this paper proposes a new classification method of complex attribute big data based on iterative fuzzy clustering algorithm. Firstly, principal component analysis and kernel local Fisher discriminant analysis were used to reduce dimensionality of complex attribute big data. Then, the Bloom Filter data structure is introduced to eliminate the redundancy of the complex attribute big data after dimensionality reduction. Secondly, the redundant complex attribute big data is classified in parallel by iterative fuzzy clustering algorithm, so as to complete the complex attribute big data classification. Finally, the simulation results show that the accuracy, the normalized mutual information index and the Richter’s index of the proposed method are close to 1, the classification accuracy is high, and the RDV value is low, which indicates that the proposed method has high classification effectiveness and fast convergence speed.

Succinct range filters

We present the Succinct Range Filter (SuRF), a fast and compact data structure for approximate membership tests. Unlike traditional Bloom filters, SuRF supports both single-key lookups and common range queries, such as range counts. SuRF is based on a new data structure called the Fast Succinct Trie (FST) that matches the performance of state-of-the-art order-preserving indexes, while consuming only 10 bits per trie node---a space close to the minimum required by information theory. Our experiments show that SuRF speeds up range queries in a widely used database storage engine by up to 5×.

Bloom Filter Encryption and Applications to Efficient Forward-Secret 0-RTT Key Exchange

Forward secrecy is considered an essential design goal of modern key establishment (KE) protocols, such as TLS 1.3, for example. Furthermore, efficiency considerations such as zero round-trip time (0-RTT), where a client is able to send cryptographically protected payload data along with the very first KE message, are motivated by the practical demand for secure low-latency communication. For a long time, it was unclear whether protocols that simultaneously achieve 0-RTT and full forward secrecy exist. Only recently, the first forward-secret 0-RTT protocol was described by Günther et al. (Eurocrypt, 2017). It is based on puncturable encryption. Forward secrecy is achieved by “puncturing” the secret key after each decryption operation, such that a given ciphertext can only be decrypted once (cf. also Green and Miers, S&P 2015). Unfortunately, their scheme is completely impractical, since one puncturing operation takes between 30 s and several minutes for reasonable security and deployment parameters, such that this solution is only a first feasibility result, but not efficient enough to be deployed in practice. In this paper, we introduce a new primitive that we term Bloom filter encryption (BFE), which is derived from the probabilistic Bloom filter data structure. We describe different constructions of BFE schemes and show how these yield new puncturable encryption mechanisms with extremely efficient puncturing. Most importantly, a puncturing operation only involves a small number of very efficient computations, plus the deletion of certain parts of the secret key, which outperforms previous constructions by orders of magnitude. This gives rise to the first forward-secret 0-RTT protocols that are efficient enough to be deployed in practice. We believe that BFE will find applications beyond forward-secret 0-RTT protocols.

Enhancing the Performance of Flow Classification in SDN-Based Intelligent Vehicular Networks

Intelligent vehicular networks converged with software-defined networking provides several flow-based surveillance services to mobile applications on vehicular nodes. But, as the scale of such networks grows exponentially, a substantial delay in processing tremendous flows emerges. The delay can be reduced by accelerating the packet classification methods, which are nowadays exploited in software-defined vehicular networks. Fast packet classification lets firewalls to inspect each incoming packet at wire speed. One of the well-known packet classification methods is the KD-tree algorithm. This paper presents an enhanced version of this algorithm that uses the geometric space to display different fields and increases search speed by recursive decomposition of the search space. Also, the enhanced KD-tree is integrated with a leaf-pushing technique, which enhances the performance of KD-tree search during classification. The proposed algorithm is implemented using a bloom filter data structure and a hash table. Experimental results show that the proposed leaf-pushed KD-tree algorithm improves packet classification speed up to 24 times in comparison with the conventional KD-tree. Moreover, the proposed algorithm can significantly reduce the classification time in comparison with state-of-the-art tree-based algorithms.

Succinct Range Filters

We present the Succinct Range Filter (SuRF), a fast and compact data structure for approximate membership tests. Unlike traditional Bloom filters, SuRF supports both single-key lookups and common range queries: open-range queries, closed-range queries, and range counts. SuRF is based on a new data structure called the Fast Succinct Trie (FST) that matches the point and range query performance of state-of-the-art order-preserving indexes, while consuming only 10 bits per trie node. The false-positive rates in SuRF for both point and range queries are tunable to satisfy different application needs. We evaluate SuRF in RocksDB as a replacement for its Bloom filters to reduce I/O by filtering requests before they access on-disk data structures. Our experiments on a 100-GB dataset show that replacing RocksDB’s Bloom filters with SuRFs speeds up open-seek (without upper-bound) and closed-seek (with upper-bound) queries by up to 1.5× and 5× with a modest cost on the worst-case (all-missing) point query throughput due to slightly higher false-positive rate.

Adaptively Secure and Fast Processing of Conjunctive Queries over Encrypted Data

This paper concerns the fundamental problem of processing conjunctive queries that contain both keyword and range conditions on public clouds in a privacy preserving manner. No prior Searchable Symmetric Encryption (SSE) based privacy preserving conjunctive query processing scheme satisfies the three requirements of adaptive security, efficient query processing, and scalable index size. In this paper, we propose the first privacy preserving conjunctive query processing scheme that satisfies all the above three requirements. To achieve adaptive security, we propose an Indistinguishable Bloom Filter (IBF) data structure for indexing.To achieve efficient query processing and structural indistinguishability, we propose a highly balanced binary tree data structure called Indistinguishable Binary Tree (IBtree). To achieve scalable and compact index size, we propose an IBtree space compression algorithm to remove redundant information in IBFs. To optimize search efficiency, we propose a traversal minimization algorithm. To make our scheme dynamic, we propose update algorithms.We prove that our scheme is adaptive secure under the IND-CKA secure model. We implemented our scheme in C++ and evaluated its performance on two real-world data sets. Experimental results show that our scheme is both fast and scalable. For example, processing a query only takes a few milliseconds for millions of records.

Puncturable Signatures and Applications in Proof-of-Stake Blockchain Protocols

Proof-of-stake blockchain protocols are becoming one of the most promising alternatives to the energy-consuming proof-of-work protocols. However, one particularly critical threat in the PoS setting is the well-known long-range attacks caused by secret key leakage (LRSL attack). Specifically, an adversary can attempt to control/compromise accounts possessing substantial stake at some past moment such that double-spend or erase past transactions, violating the fundamental persistence property of blockchain. Puncturable signatures provide a satisfying solution to construct practical proof-of-stake blockchain resilient to LRSL attack, despite of the fact that existent constructions are not efficient enough for practical deployments. In this paper, we provide an in-depth study of puncturable signatures and explore its applications in the proof-of-stake blockchain. We formalize a security model that allows the adversary for adaptive signing and puncturing queries, and show a construction with efficient puncturing operations based on the Bloom filter data structure and strong Diffie-Hellman assumption. The puncturing functionality we desire is for a particular part of message, like prefix, instead of the whole message. Furthermore, we use puncturable signatures to construct practical proof-of-stake blockchain protocols that are resilient to LRSL attack, while previously the forward-secure signature is used to immunize this attack. We implement our scheme and provide experimental results showing that in comparison with the forward-secure signature, our construction performs substantially better on signature size, signing and verification efficiency, significantly on key update efficiency.

Succinct Range Filters

We present the Succinct Range Filter (SuRF), a fast and compact data structure for approximate membership tests. Unlike traditional Bloom filters, SuRF supports both single-key lookups and common range queries. SuRF is based on a new data structure called the Fast Succinct Trie (FST) that matches the point and range query performance of state-of-the-art order-preserving indexes, while consuming only 10 bits per trie node. The false positive rates in SuRF for both point and range queries are tunable to satisfy different application needs. We evaluate SuRF in RocksDB as a replacement for its Bloom filters to reduce I/O by filtering requests before they access on-disk data structures. Our experiments on a 100 GB dataset show that replacing RocksDB's Bloom filters with SuRFs speeds up open-seek (without upper-bound) and closed-seek (with upper-bound) queries by up to 1.5× and 5× with a modest cost on the worst-case (all-missing) point query throughput due to slightly higher false positive rate.

Probabilistic Properties of the Spatial Bloom Filters and Their Relevance to Cryptographic Protocols

The classical Bloom filter data structure is a crucial component of hundreds of cryptographic protocols. It has been used in privacy preservation and secure computation settings, often in conjunction with the (somewhat) homomorphic properties of ciphers such as Paillier's. In 2014, a new data structure extending and surpassing the capabilities of the classical Bloom filter has been proposed. The new primitive, called spatial Bloom filter (SBF) retains the hash-based membership-query design of the Bloom filter, but applies it to elements from multiple sets. Since its introduction, the SBF has been used in the design of cryptographic protocols for a number of domains, including location privacy and network security. However, due to the complex nature of this probabilistic data structure, its properties had not been fully understood. In this paper, we address this gap in knowledge and we fully explore the probabilistic properties of the SBF. In doing so, we define a number of metrics (such as emersion and safeness) useful in determining the parameters needed to achieve certain characteristics in a filter, including the false positive probability and inter-set error rate. This will in turn enable the design of more efficient cryptographic protocols based on the SBF, opening the way to their practical application in a number of security and privacy settings.

Reducing false rate packet recognition using Dual Counting Bloom Filter

Distributed Denial of Service (DDoS) attacks are a serious threat to Internet security. A lot of research effort focuses on having detection and prevention methods on the victim server side or source side. The Bloom filter is a space-efficient data structure used to support pattern matching problems. The filter is utilised in network applications for deep packet inspection of headers and contents and also looks for predefined strings to detect irregularities. In intrusion detection systems, the accuracy of pattern matching algorithms is crucial for dependable detection of matching pairs, and its complexity usually poses a critical performance bottleneck. In this paper, we will propose a novel Dual Counting Bloom Filter (DCBF) data structure to decrease false detection of matching packets applicable for the $$\textit{SACK}^2$$ algorithm. A theoretical evaluation will determine the false rate probability of detection and requirements for increased memory. The proposed approach significantly reduces the false rate compared to previously published results. The results indicate that the increased complexity of the DCBF does not affect efficient implementation of hardware for embedded systems that are resource constrained. The experimental evaluation was performed using extensive simulations based on real Internet traces of a wide area network link, and it was subsequently proved that DCBF significantly reduces the false rate.

Quaestor

Today, web performance is primarily governed by round-trip latencies between end devices and cloud services. To improve performance, services need to minimize the delay of accessing data. In this paper, we propose a novel approach to low latency that relies on existing content delivery and web caching infrastructure. The main idea is to enable application-independent caching of query results and records with tunable consistency guarantees, in particular bounded staleness. Q uaestor (Query Store) employs two key concepts to incorporate both expiration-based and invalidation-based web caches: (1) an Expiring Bloom Filter data structure to indicate potentially stale data, and (2) statistically derived cache expiration times to maximize cache hit rates. Through a distributed query invalidation pipeline, changes to cached query results are detected in real-time. The proposed caching algorithms offer a new means for data-centric cloud services to trade latency against staleness bounds, e.g. in a database-as-a-service. Q uaestor is the core technology of the backend-as-a-service platform Baqend, a cloud service for low-latency websites. We provide empirical evidence for Q uaestor 's scalability and performance through both simulation and experiments. The results indicate that for read-heavy workloads, up to tenfold speed-ups can be achieved through Q uaestor 's caching.

An Anonymous Inter-Network Routing Protocol for the Internet of Things

With the diffusion of the Internet of Things (IoT), computing is becoming increasingly pervasive, and different heterogeneous networks are integrated into larger systems. However, as different networks managed by different parties and with different security requirements are interconnected, security becomes a primary concern. IoT nodes, in particular, are often deployed “in the open”, where an attacker can gain physical access to the device. As nodes can be deployed in unsurveilled or even hostile settings, it is crucial to avoid escalation from successful attacks on a single node to the whole network, and from there to other connected networks. It is therefore necessary to secure the communication within IoT networks, and in particular, maintain context information private, including the network topology and the location and identity of the nodes. In this paper, we propose a protocol achieving anonymous routing between different interconnected networks, designed for the Internet of Things and based on the spatial Bloom filter (SBF) data structure. The protocol enables private communication between the nodes through the use of anonymous identifiers, which hide their location and identity within the network. As routing information is encrypted using a homomorphic encryption scheme, and computed only in the encrypted domain, the proposed routing strategy preserves context privacy, preventing adversaries from learning the network structure and topology. This, in turn, significantly reduces their ability to gain valuable network information from a successful attacks on a single node of the network, and reduces the potential for attack escalation.

An Anonymous Inter-Network RoutingProtocol for the Internet of Things

With the diffusion of the Internet of Things (IoT), computing is becoming increasingly pervasive, and different heterogeneous networks are integrated into larger systems. However, as different networks managed by different parties and with different security requirements are interconnected, security becomes a primary concern. IoT nodes, in particular, are often deployed “in the open”, where an attacker can gain physical access to the device. As nodes can be deployed in unsurveilled or even hostile settings, it is crucial to avoid escalation from successful attacks on a single node to the whole network, and from there to other connected networks. It is therefore necessary to secure the communication within IoT networks, and in particular, maintain context information private, including the network topology and the location and identity of the nodes. In this paper, we propose a protocol achieving anonymous routing between different interconnected networks, designed for the Internet of Things and based on the spatial Bloom filter (SBF) data structure. The protocol enables private communication between the nodes through the use of anonymous identifiers, which hide their location and identity within the network. As routing information is encrypted using a homomorphic encryption scheme, and computed only in the encrypted domain, the proposed routing strategy preserves context privacy, preventing adversaries from learning the network structure and topology. This, in turn, significantly reduces their ability to gain valuable network information from a successful attacks on a single node of the network, and reduces the potential for attack escalation.

An Efficient Broadcast Authentication Scheme in Wireless Sensor Networks

Abstract: Broadcast source authentication is a challenging topic in wireless sensor networks. This security service allows senders to broadcast messages to multiple receivers in a secure way. Although several authentication mechanisms have been proposed to address the need for security in WSNs, most of them are resource consuming and are inadequate for constrained environments. In this paper, we shed the light to the security vulnerabilities of symmetric key based authentication mechanisms, and their inability to tackle memory DoS attacks. Moreover, we provide a new efficient broadcast authentication scheme based on a Bloom filter data structure in order to reduce the communication overhead. Finally, we run a thorough set of simulations to assess the efficiency of our approach compared to some state of the art solutions in terms of energy consumption, communication and computation overhead. Our results provide insight into the suitability of our approach for use in WSNs.

A novel authentication and access control framework in wireless sensor networks

Wireless Sensor Networking continues to evolve as one of the most challenging research areas. Considering the insecure nature of these networks and the fact that sensor nodes are distributed in a hostile environment, having a well-implemented security scheme is absolutely essential. Bearing in mind the important security services like authentication and access control, we have proposed a novel security framework for these networks. The new framework is based on Kerberos authentication and access control system. The Kerberos has been adopted for WSNs by utilizing Bloom Filter data structure and Elliptic Curve cryptography. In the proposed scheme, Bloom Filter data structure is used in a novel way; we have used this data structure to get rid of Public Key’s certificates. By combining Bloom Filter data structure and Elliptic Curve cryptography, we achieved a very light robust security framework that offers Authentication, Access Control, and key sharing services. The analysis results showed that our scheme provides more security services and is more robust in the presence of attacks compared to the previous schemes. In contrast, simulation results indicated that our system had significant improvements over the other schemes in many aspects such as power and time expenditure.

BSI: Bloom Filter-Based Semantic Indexing for Unstructured P2P Networksh

Resource management and search is very important yet challenging in large-scale distributed systems like P2Pnetworks. Most existing P2P systems rely on indexing to efficiently route queries over the network. However, searches based on such indices face two key issues. First, majority of existing search schemes often rely on simply keyword based indices that can only support exact string based matches without taking into account the meaning of words. Second it is difficult, if not impossible, to devise query based indexing schemes that can represent all possible concept combinations without resulting in exponential index sizes. To address these problems, we present BSI, a novel P2P indexing and query routing strategy to support semantic based content searches. The BSI indexing structure captures the semantic content of documents using a reference ontology. Our indexing scheme can efficiently handle multi-concept queries by maintaining summary level information for each individual concept and concept combinations using a novel space-efficient Two-level Semantic Bloom Filter(TSBF) data structure. By using TSBFs to represent a large document and query base, BSI significantly reduces the communication cost and storage cost of indices. Furthermore, We devise a low-overhead mechanism to allow peers to dynamically estimate the relevance strength of a peer for multi-concept queries with high accuracy solely based on TSBFs. We also propose a routing index compression mechanism to observe peers’ dynamic storage limitations with minimal loss of information by exploiting a reference ontology structure. Based on the proposed index structure, we design a novel query routing algorithm that exploits semantic based information to route queries to semantically relevant peers. Performance evaluation demonstrates that our proposed approach can improve the search recall of unstructured P2P systems up to 383.71% while keeping the communication cost at a low level compared to state-of-art search mechanism OSQR [7].