Bloom Filter Research Articles

ZIP: Lazy Imputation during Query Processing

This paper develops a query-time missing value imputation framework, entitled ZIP, that modifies relational operators to be imputation aware in order to minimize the joint cost of imputing and query processing. The modified operators use a cost-based decision function to determine whether to invoke imputation or to defer to downstream operators to resolve missing values. The modified query processing logic ensures results with deferred imputations are identical to those produced if all missing values were imputed first. ZIP includes a novel outer-join based approach to preserve missing values during execution, and a bloom filter based index to optimize the space and running overhead. Extensive experiments on both real and synthetic data sets demonstrate 10 to 25 times improvement when augmenting the state-of-the-art technology, ImputeDB, with ZIP-based deferred imputation. ZIP also outperforms the offline approach by up to 19607 times in a real data set.

PSI-CA-Based Vehicle Selection Scheme for Data Sharing in Internet of Vehicles

In recent years, the development of the Internet of Vehicles (IoV) has led to an increase in the demand for data sharing services for the IoV. In the era of big data, safe and convenient data sharing services for IoV are inseparable from the support of reliable data. With the increase in the number of smart cars, how to filter the data provided by vehicles while protecting privacy to improve the quality of data sharing services has attracted the attention of scholars. However, in the existing data sharing scheme for IoV, the vehicles that provide data are selected based on the reputation mechanism or the voting mechanism. Our analysis shows that these two mechanisms lack objectivity and are vulnerable to cooperative attack. Therefore, based on spatiotemporal matching, a new trusted relationship establishment method, this paper proposes an objective and cooperative attack-resistant vehicle selection scheme for data sharing in IoV. Bloom filters and an exponential ElGamal encryption scheme are used to implement private set intersection cardinality (PSI-CA) technology for evaluating spatiotemporal matching level. In this paper, the scheme proposed is of great significance to improve existing vehicle services and improve driving safety.

Two-layer partitioned and deletable deep bloom filter for large-scale membership query

The recently proposed Learned Bloom Filter (LBF) provides a new perspective on large-scale membership queries by using machine learning to replace the traditional bloom filter. However, reducing the false positive rate (FPR) of the learned model with small memory usage, and supporting deletion efficiently become the new issues. In this paper, we propose a novel Two-layer Partitioned and Deletable Deep Bloom Filter (PDDBF) for large-scale membership query, which can reduce the FPR with small memory usage and support deletion efficiently. The proposed PDDBF consists of three main parts: (1) Data partition. To improve the classification accuracy of the learned model, the K-means cluster with the elbow method is used for the data partition. (2) Deep Bloom Filter. To reduce the FPR, deep learning models are used to construct multiple independent learning mechanisms, which correspond to the clusters obtained by part1. (3) Partitioned backup filter. To support deletion under the premise of ensuring low FPR and reducing query time consumption, combine the perfect hash (PH) table and counting bloom filters (CBFs) on the basis of the partition bloom filter. Experiments show that the proposed PDDBF reduces the FPR 87.13% with the same memory usage compared with the state-of-the-art PLBF on real-world URLs data set. Moreover, the PDDBF reduces the FPR 99.68% with the same memory usage and reduces the query time consumption to 2.61x that of the PLBF after data deletion, respectively.

Fine-grained access control policy in blockchain-enabled edge computing

In the process of data sharing and interaction between untrusted intelligence edge devices, there are risks of misuse and tampering with the data. Hence, how to ensure data security and prevent unauthorized data access is the key problem to be solved in the cloud edge collaborative architecture. However, due to the heterogeneity of edge devices and the diversity of data sources, traditional access control methods cannot meet the requirements of the large-scale, distributed, dynamic and flexible expansion of edge intelligence applications. Therefore, this paper studies the fine-grained data access control method on the basis of smart contracts, and implements the attribute-based access control (ABAC) model through smart contracts. To improve the query efficiency of attributes and policies in the process of access request decision, an attribute management scheme is designed on the basis of the Bloom filter to achieve fast retrieval of attributes and policies. The decentralized fine-grained access control method is tested under the performance indicators such as retrieval delay, deployment delay, rule reduction rate and decision response time.

SNN-PPRL: A secure record matching scheme based on siamese neural network

With the rapid development of IoT, data from various devices is increasing exponentially. To better apply and analyze IoT data, it is necessary to integrate them from different parties. However, how to match the same entity’s records in huge amounts of data accurately and ensure data privacy has become an urgent problem to be solved. Therefore, privacy preserving record linkage technology is proposed. However, the existing method often has problems in adaptability and linkage quality. We propose the first privacy preserving record linkage scheme based on deep learning techniques. In our scheme, a combined bloom filter module is proposed to encode the data, which can improve security and protect privacy. At the same time, by introducing the record matching module based on siamese neural network, the adaptability and linkage quality are improved. Experiments show that SNN-PPRL can achieve up to 94% matching accuracy and 99% recall.

CRBF: Cross-Referencing Bloom-Filter-Based Data Integrity Verification Framework for Object-Based Big Data Transfer Systems

Various components are involved in the end-to-end path of data transfer. Protecting data integrity from failures in these intermediate components is a key feature of big data transfer tools. Although most of these components provide some degree of data integrity, they are either too expensive or inefficient in recovering corrupted data. This problem highlights the need for application-level end-to-end integrity verification during data transfer. However, the computational, memory, and storage overhead of big data transfer tools can be a significant bottleneck for ensuring data integrity due to the large size of the data. This paper proposes a novel framework for data integrity verification in big data transfer systems using a cross-referencing Bloom filter. This framework has three advantages over state-of-the-art data integrity techniques: lower computation and memory overhead and zero false-positive errors for a limited number of elements. This study evaluates the computation, memory, recovery time, and false-positive overhead for the proposed framework and compares them with state-of-the-art solutions. The evaluation results indicate that the proposed framework is efficient in detecting and recovering from integrity errors while eliminating false positives in the Bloom filter data structure. In addition, we observe negligible computation, memory, and recovery overheads for all workloads.

An Efficient Encryption Scheme with Fully Hidden Access Policy for Medical Data

With the development of informatization, patient medical data are generally stored in the form of electronic medical records. Hospitals store a large amount of medical data on cloud servers, which can ensure the integrity and sharing of medical data. There are many problems when sharing medical data, the most important of which is the security of private data. The ciphertext-policy attribute-based encryption (CP-ABE) algorithm not only supports data encryption, but also supports data access control. The CP-ABE algorithm is applied to medical databases. Through the encryption of medical data and the fine-grained control of data owner access to medical data, the privacy security and sharing security of medical data are realized. However, the traditional CP-ABE algorithm also has problems for the data user regarding access policy privacy leakage and the low efficiency of medical data encryption and decryption. This paper proposes a secure and efficient encryption technique. Medical data are encrypted using a symmetric encryption algorithm, and the CP-ABE algorithm with a fully hidden access policy encrypts the symmetric key. To implement access policy hiding, it uses an attribute bloom filter (ABF) for all access structures. The solution also supports the outsourced decryption of medical data, which can improve the encryption and decryption efficiency of the technique. Finally, in this paper, it is demonstrated that the scheme is selectively chosen-plaintext attack secure (CPA-secure). The experimental results show that users have less computational consumption to obtain medical data.

EAQ: Enabling Authenticated Complex Query Services in Sustainable-Storage Blockchain

The data query service is urgently required in sustainable-storage blockchain, where full nodes store the entire transaction data while light nodes only store block headers. Queries invariably seek data with multiple attributes. However, no existing method provides a unified authenticated data structure (ADS) to support complex query operators (e.g., range queries and data object queries) on multiattribute blockchain data. In this paper, we propose a framework EAQ that effectively supports both fast data queries on multiple attributes and authentication of the query result. We propose a new ADS, called the MR <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\mathit{Bloom}$</tex-math></inline-formula> -tree, based on the Bloom filter (BF) and Merkle R-tree. We prove the decomposability of BFs, which enables the BF to be seamlessly incorporated with the Merkle R-tree. This ADS enables range-level search using multidimensional attribute ranges and object-level search using BFs. This ADS also supports querying and proving inexistent data objects. To reduce storage overhead, we improve the MR <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\mathit{Bloom}$</tex-math></inline-formula> -tree using the suppressed BF structure, which constructs only one BF independent of the number of attributes. To manage string attributes, we transform them into discrete numerical attributes using density-based clustering to represent similar items with close numerical values. Experiments show that the proposed framework achieves promising results.

Attacks and Improvement of Unlinkability of Biometric Template Protection Scheme Based on Bloom Filters

Biometric technologies are being prominently used everywhere. However, the leakage of biometric information can pose a serious security risk, making the protection of biometric templates particularly important and receiving more attention. Rathgeb et al. first proposed the cancelable biometric technology based on Bloom filters in 2013, which has been applied to protect different biometric templates. Bloom filter-based biometrics offer the advantages of alignment-free, fast recognition and high accuracy. An ideal biometric system should also be irreversibility and unlinkability. In this paper, firstly, we propose a reverse reconstruction attack. Through the reverse reconstruction of Bloom filters, we find that the reconstructed biometric data and the original biometric data have some strong statistical correlation, which proves that the scheme has the linkability defect. Experiments show that for the original Bloom filter-based biometric template protection scheme, we can judge whether two different biometric templates belong to the same user with a success probability of 71.0%. Secondly, to remedy above defect, we construct a structure-preserving encryption scheme, i.e., the feature template encrypted with it maintains the structure and length of the original template, making it impossible for an attacker to reconstruct meaningful data from Bloom filter. Finally, an improved biometric template protection scheme based on Bloom filters is proposed by introducing the proposed encryption. Attack experiment shows that the improved scheme can effectively resist the reverse reconstruction attack, with the success probability of attacking the unlinkability of the improved scheme being 50.0%, which is the same as the probability of random guessing. Performance evaluation shows that the proposed scheme maintains the biometric performance of the original system and the unprotected system.

Space-Efficient Storage Structure of Blockchain Transactions Supporting Secure Verification

The rapid growth of the blockchain size is a major bottleneck hindering its implementations in data-heavy applications. Current efforts improve the distributed storage ways and transactions' storage mechanisms of blockchain, however, the blockchain distribution and integrity are destroyed. Simplified Payment Verification (SPV) is closely related to blockchain storage, but the current solutions did not explore the privacy-preserving SPV. In this paper, we propose a new storage structure for blockchain transactions, called Coloring Index (CI), to reduce the blockchain's space occupation. Specifically, we devise an index building algorithm to simply calculate the indices of transactions for the sake of information concealing. By improving the Coloring Embedder for multi-sets query, we can store the indices into the Coloring Embedder to achieve the structured storage of transactions with small space occupation. Using CI, SPV query proceeds without revealing the user's address, thereby achieving secure data sharing in applications such as the intelligent vehicles' communications and distributed IoT. We prove CI's security against malicious full nodes when establishing possible connections between the address and the user. The experiments show that blockchain systems using our CI store one time more transactions than Merkle tree and half more than Bloom filter.

BPMS: Blockchain-Based Privacy-Preserving Multi-Keyword Search in Multi-Owner Setting

Searchable encryption&#x00A0;(SE) has emerged as a cryptographic primitive that allows data users to search on encrypted data. Most existing SE schemes usually delegate search operations to an intermediary such as a cloud server, which would inevitably result in single-point failure, privacy leakage, and even untrustworthy results. Several blockchain-based SE schemes have been proposed to alleviate these issues; however, they suffer from some issues, such as the support for multi-keyword multi-owner model, query privacy and data storage availability. In this paper, we propose BPMS, blockchain-based privacy-preserving multi-keyword search in multi-owner setting, which supports searching over encrypted data in trustworthy, private and efficient manners. The attribute Bloom filter has been introduced into our BPMS to build indexes, which protects query privacy and improves index generation performance. To guarantee data storage availability, our BPMS leverages the advantages of IPFS&#x00A0;(InterPlanetary File System) to store large scale of encrypted data. Security proof and comparative analysis in theory indicate that our BPMS is more secure and efficient. A series of experiments conducted on a real-world dataset further demonstrate that our BPMS is feasible in practice.

Nereus: Anonymous and Secure Ride-Hailing Service Based on Private Smart Contracts

Security and privacy issues have become a major hindrance to the broad adoption of Ride-Hailing Services (RHSs). In this paper, we introduce a new collusion attack initiated by the Ride-Hailing Service Provider (RHSP) and a driver that could easily link the real riders and their anonymous requests (credentials). Besides this attack, existing work requires heavy computations to execute user matching, and it is challenging for riders to verify matching results. Meanwhile, a malicious driver may cancel an assigned ride order due to its short distance. To address these issues, we present a RHS system named Nereus to support collusion resistance, efficiency, verifiability, and accountability. First, we integrate a smart contract into a Software Guard Extensions (SGX) enclave to establish a <i>private smart contract</i> for collusion resistance. We use a Bloom filter to achieve efficient matching. Second, we leverage privacy-preserving range query and Merkle proofs to make matching results verifiable. Meanwhile, we adopt short group signatures to provide anonymous authentication and deposit commitments to hold the runaway driver accountable. We formally state and prove the security and privacy of Nereus. We build a prototype based on Ethereum and SGX to conduct extensive performance analysis in regard to gas costs, computational costs, and communication overhead. Experimental results show that Nereus significantly improves over existing schemes in terms of computational costs.

BeDCV: Blockchain-Enabled Decentralized Consistency Verification for Cross-Chain Calculation

With the increase of data stored on the blockchain, the efficiency of storage and calculation of blockchain has gradually become a bottleneck restricting the development of blockchain. By storing data on multiple chains, blockchains can request data from other chains for calculation and the storage pressure can be alleviated. But the transfer of a large amount of data between chains suffers from low transfer efficiency and poor security. A reasonable design is to perform the calculation on the data storage chain and only transfer the results across chains. However, since the calculation process is invisible, blockchains cannot judge the consistency of calculation results from other chains. In this paper, we provide a blockchain-enabled decentralized consistency verification scheme for cross-chain calculation (BeDCV). Considering the decentralized characteristic of blockchain, we adopt the blockchain called <i>supervision chain</i> for decentralized auditing. We modify paillier homomorphic encryption to encrypt data involved in the calculation for correctness verification. Then, we aggregate the ciphertexts of data to generate the audit proof for integrity verification. Besides, we verify whether the data involved in the calculation are real-time by leveraging a counting bloom filter. The supervision chain can check the correctness, integrity, and real-time performance of cross-chain data calculation without revealing any original information about the data. The theoretical and experimental analysis demonstrates that BeDCV can verify the consistency of cross-chain data calculation result effectively, realizing secure and reliable expansion of blockchain.

A Vulnerability Assessment Framework for Privacy-preserving Record Linkage

The linkage of records to identify common entities across multiple data sources has gained increasing interest over the last few decades. In the absence of unique entity identifiers, quasi-identifying attributes such as personal names and addresses are generally used to link records. Due to privacy concerns that arise when such sensitive information is used, privacy-preserving record linkage (PPRL) methods have been proposed to link records without revealing any sensitive or confidential information about these records. Popular PPRL methods such as Bloom filter encoding, however, are known to be susceptible to various privacy attacks. Therefore, a systematic analysis of the privacy risks associated with sensitive databases as well as PPRL methods used in linkage projects is of great importance. In this article we present a novel framework to assess the vulnerabilities of sensitive databases and existing PPRL encoding methods. We discuss five types of vulnerabilities: frequency, length, co-occurrence, similarity, and similarity neighborhood, of both plaintext and encoded values that an adversary can exploit in order to reidentify sensitive plaintext values from encoded data. In an experimental evaluation we assess the vulnerabilities of two databases using five existing PPRL encoding methods. This evaluation shows that our proposed framework can be used in real-world linkage applications to assess the vulnerabilities associated with sensitive databases to be linked, as well as with PPRL encoding methods.

A Review of Cuckoo Filters for Privacy Protection and Their Applications

As the global digitalization process continues, information is transformed into data and widely used, while the data are also at risk of serious privacy breaches. The Cuckoo filter is a data structure based on the Cuckoo hash. It encrypts data when it is used and can achieve privacy protection to a certain extent. The Cuckoo filter is an alternative to the Bloom filter, with advantages such as support for deleting elements and efficient space utilization. Cuckoo filters are widely used and developed in the fields of network engineering, storage systems, databases, file systems, distributed systems, etc., because they are often used to solve collection element query problems. In recent years, many variants of the Cuckoo filter have emerged based on ideas such as improving the structure and introducing new technologies in order to accommodate a variety of different scenarios, as well as a huge collection. With the development of the times, the improvement of the structure and operation logic of the Cuckoo filter itself has become an important direction for the research of aggregate element query.

Seismic Data Query Algorithm Based on Edge Computing

Edge computing can reduce the transmission pressure of wireless networks in earthquakes by pushing computing functionalities to network edges and avoiding the data transmission to cloud servers. However, this also leads to the scattered storage of data content in each edge server, increasing the difficulty of content search. This paper investigates the seismic data query problem supported by edge computing. We first design a lookup mechanism based on bloom filter, which can quickly determine if there is the information that we need on a particular edge server. Then, the MEC-based data query problem is formulated as an optimization problem whose goal is to minimize the long-term average task delay with the constraints of computing capacity of edge servers. To reduce the complexity of problem, we further transform it as a Markov Decision Process by defining state space, action space and reward function. A novel DQN-based seismic data query algorithm is proposed to solve problem effectively. Extensive simulation-based testing shows that the proposed algorithm performances better when compared with two state-of-the-art solutions.

A Framework for Privacy Preserving Localized Graph Pattern Query Processing

This paper studies privacy preserving graph pattern query services in a cloud computing paradigm. In such a paradigm, data owner stores the large data graph to a powerful cloud hosted by a service provider (SP) and users send their queries to SP for query processing. However, as SP may not always be trusted, the sensitive information of users' queries, importantly, the query structures, should be protected. In this paper, we study how to outsource the localized graph pattern queries (LGPQs) on the SP side with privacy preservation. LGPQs include a rich set of semantics, such as subgraph homomorphism, subgraph isomorphism, and strong simulation, for which each matched graph pattern is located in a subgraph called ball that have a restriction on its size. To provide privacy preserving query service for LGPQs, this paper proposes the first framework, called Prilo, that enables users to privately obtain the query results. To further optimize Prilo, we propose Prilo* that comprises the first bloom filter for trees in the trust execution environment (TEE) on SP, a query-oblivious twiglet-based technique for pruning non-answers, and a secure retrieval scheme of balls that enables user to obtain query results early. We conduct detailed experiments on real world datasets to show that Prilo* is on average 4x faster than the baseline, and meanwhile, preserves query privacy.

MAD‐DDS: Memory‐efficient automatic discovery data distribution service for large‐scale distributed control network

AbstractThe rampant deployment of data distribution service (DDS) as middle‐ware service providers for industrial network platforms has been widely investigated. All DDS‐based node discovery protocols establish communication with its intended target systems by accessing matched endpoints/nodes information. These matched endpoints/nodes information is usually embedded with the system’s programmable control plane network which acts as the conveyor vehicle. The introduction of software defined networking (SDN) is to characterize the control plane from embedded data plane. The DDS implements the simple discovery protocol (SDP) as its inherent node discovery protocol. Deploying DDS for data packet exchange in server‐based collaborative distributed networked control (DNC) systems has gained traction. The current automatic discovery protocol (ADP) based on SDP is fraught with real‐time limitations such as high memory consumption and poor packet transmission. This work presents novel memory‐efficient automatic discovery data distribution service (MAD‐DDS) with enhanced threshold bloom filters (ETBF) where ETBF stores transmission packets at simulation end‐nodes. The packet is further adjusted using optimized binarization and decision thresholds inside ADP, hence guaranteeing memory reduction. The testbed computation recorded significantly improved quality of service (QoS) whereas numerical results depict significant decline in memory consumption with consistent packet transmission rates that produces increased computational capacity.

Lightweight Transmission Behavior Audit Scheme for NDN Industrial Internet Identity Resolution and Transmission Based on Blockchain

The Named Data Network (NDN) enables efficient content dissemination through interest-based retrieval, name-based routing, and content caching. In the industrial Internet architecture based on NDN, device identity distribution, identification, resolution, and routing rely on identification resolution technology. However, this approach presents challenges such as cache poisoning, interest packet flood attacks, and black hole attacks. Existing security schemes primarily focused on routing forwarding and verification fail to address critical concerns, including routing environment credibility and data leakage, while exhibiting poor time and space efficiency. To address these challenges, this paper proposes a lightweight behavior auditing scheme using blockchain technology. The scheme utilizes an improved Bloom filter to compress behavioral information like interest and data packets during the identification transmission process. The compressed data are subsequently uploaded to a blockchain for auditing, achieving efficient space and time utilization while maintaining feasibility.

Finger Vein Template Protection with Directional Bloom Filter

Biometrics has become a widely accepted solution for secure user authentication. However, the use of biometric traits raises serious concerns about the protection of personal data and privacy. Traditional biometric systems are vulnerable to attacks due to the storage of original biometric data in the system. Because biometric data cannot be changed once it has been compromised, the use of a biometric system is limited by the security of its template. To protect biometric templates, this paper proposes the use of directional bloom filters as a cancellable biometric approach to transform the biometric data into a non-invertible template for user authentication purposes. Recently, Bloom filter has been used for template protection due to its efficiency with small template size, alignment invariance, and irreversibility. Directional Bloom Filter improves on the original bloom filter. It generates hash vectors with directional subblocks rather than only a single-column subblock in the original bloom filter. Besides, we make use of multiple fingers to generate a biometric template, which is termed multi-instance biometrics. It helps to improve the performance of the method by providing more information through the use of multiple fingers. The proposed method is tested on three public datasets and achieves an equal error rate (EER) as low as 5.28% in the stolen or constant key scenario. Analysis shows that the proposed method meets the four properties of biometric template protection. Doi: 10.28991/HIJ-2023-04-02-013 Full Text: PDF