Lookup Time Research Articles

A Closer Look into IPFS: Accessibility, Content, and Performance

The InterPlanetary File System (IPFS) has recently gained considerable attention. While prior research has focused on understanding its performance characterization and application support, it remains unclear: (1) what kind of files/content are stored in IPFS, (2) who are providing these files, (3) are these files always accessible, and (4) what affects the file access performance. To answer these questions, in this paper, we perform measurement and analysis on over 4 million files associated with CIDs (content IDs) that appeared in publicly available IPFS datasets. Our results reveal the following key findings: (1) Mixed file accessibility: while IPFS is not designed for a permanent storage, accessing a non-trivial portion of files, such as those of NFTs and video streams, often requires multiple retrieval attempts, potentially blocking NFT transactions and negatively affecting the user experience. (2) Dominance of NFT (non-fungible token) and video files: about 50% of stored files are NFT-related, followed by a large portion of video files, among which about half are pirated movies and adult content. (3) Centralization of content providers: a small number of peers (top-50), mostly cloud nodes hosted by tech companies, serve a large portion (95%) of files, deviating from IPFS's intended design goal. (4) High variation of downloading throughput and lookup time: large file retrievals experience lower average throughput due to more overhead for resolving file chunk CIDs, and looking up files hosted by non-cloud nodes takes longer. We hope that our findings can offer valuable insights for (1) IPFS application developers to take into consideration these characteristics when building applications on top of IPFS, and (2) IPFS system developers to improve IPFS and similar systems to be developed for Web3.

Efficient keyword search on encrypted dynamic cloud data

Sensitive information is increasingly being outsourced to the cloud. In order to protect the privacy of such sensitive data, cloud users (clients) encrypt their data before outsourcing. However, this poses a difficulty to later perform search operations on the encrypted data. Searchable encryption schemes enable a client to search and retrieve the cloud data (based on the keywords present in the data) when the data is encrypted. Dynamic searchable encryption schemes allow the client to search over the encrypted cloud data even when new documents are added to or deleted from the encrypted data. There is a trade-off between security (that is measured in terms of information leaked to the cloud) and the efficiency of dynamic searchable encryption schemes. Stronger security guarantees often come at a cost of less efficiency.In this work, we propose a new dynamic searchable encryption scheme for cloud data that achieves better security guarantees and improved efficiency compared to popular dynamic searchable encryption schemes. Our scheme uses an efficient data structure that reduces storage, lookup (search) time, and database modification time. We build a prototype of our scheme and experiment on large real-life datasets. We show our scheme performs better than the existing schemes, which provide similar (or weaker) security.

An Efficient Pending Interest Table Content Search In NDN through Stable Bloom Filter

AbstractNamed Data Networking (NDN) has gained importance in today’s era due to a paradigm shift in the Internet usage pattern which revolves around the content rather than the respective host addresses. Three important data structures in NDN are Content Store (CS), Pending Interest Table (PIT) and Forwarding Information Base (FIB). The search time of PIT is quite high since its size grows with the addition of new content names, and the Interest packets which are not served by CS are searched in millions of existing entries in the PIT. Hence, lookup time can be improved if, instead of checking all the available entries, initial scanning is done to determine whether the required content name exists in the PIT or not. In this paper, we propose a Stable Bloom Filter (SBF) based PIT called S-PIT, to minimize the PIT search time by identifying the existence of query content through SBF. The various experiments performed show that S-PIT outperforms existing data structures in terms of memory consumption, content insertion time, average search time and false positive rate.

Improved IP lookup technology for trie-based data structures

Many Internet protocol (IP) lookup algorithms have been formulated to improve network performance. This study reviewed and experimentally evaluated technologies for trie-based methods that reduce memory access, memory consumption and IP lookup time. Experiments involving binary tries (for simplicity) were conducted on four real-world data sets, two of which were on IPv4 router tables (855,997 and 876,489 active prefixes) and two of which were on IPv6 router tables (155,310 and 157,579 active prefixes). Technologies designed to reduce time taken (at the expense of memory consumption) worked well for IPv4, and those designed to reduce memory consumption worked well for IPv6. Various combinations of these technologies were applied together in the experiments.

Multi-Agent Multi-Objective Optimization of a Very-Long-Baseline Interferometry Mission

Deep-space-distributed spacecraft missions continue to gain relevance as real-world missions such as the Laser Interferometer Space Antenna, Mars Sample Return, and others are conceived, designed, and flown. These classes of mission designs continue to challenge state-of-the-art trajectory optimization tools, often requiring significant changes to software and even new problem transcriptions entirely, in order to solve for optimal solutions. Trajectory optimization for multiple-vehicle missions poses all the challenges of the optimization of single-vehicle missions, but with added complexity of increased combinatorial scope due to multiple spacecraft, interspacecraft coordination constraints, and coordinated science objectives, motivating the need for new technical capabilities. The goal of this paper is to apply a multi-objective, multi-agent hybrid optimal control problem transcription, utilizing several new techniques, to optimize a very-long-baseline interferometry mission design. We develop and apply several new techniques in this capability, including a HashMap archive utility to prevent resolving missions, and a null gene transcription to vary both fleet size, and observation multiplicity. Applying these techniques enables efficient exploration the multi-objective nondominated front of a multivehicle design space where the number of spacecraft varies. The HashMap archive utility proves an essential component of this capability as duplicate candidate missions are discovered on average 17% of the time, and it yields an order of magnitude quicker lookup time compared with a text file analog. Within the resulting nondominated front of solution missions, many interesting solutions appear, including one mission with a fleet of 7 spacecraft that images 16 radio sources.

Comparative genomic and biochemical analyses identify a collagen galactosylhydroxylysyl glucosyltransferase from Acanthamoeba polyphaga mimivirus

Humans and Acanthamoeba polyphaga mimivirus share numerous homologous genes, including collagens and collagen-modifying enzymes. To explore this homology, we performed a genome-wide comparison between human and mimivirus using DELTA-BLAST (Domain Enhanced Lookup Time Accelerated BLAST) and identified 52 new putative mimiviral proteins that are homologous with human proteins. To gain functional insights into mimiviral proteins, their human protein homologs were organized into Gene Ontology (GO) and REACTOME pathways to build a functional network. Collagen and collagen-modifying enzymes form the largest subnetwork with most nodes. Further analysis of this subnetwork identified a putative collagen glycosyltransferase R699. Protein expression test suggested that R699 is highly expressed in Escherichia coli, unlike the human collagen-modifying enzymes. Enzymatic activity assay and mass spectrometric analyses showed that R699 catalyzes the glucosylation of galactosylhydroxylysine to glucosylgalactosylhydroxylysine on collagen using uridine diphosphate glucose (UDP-glucose) but no other UDP-sugars as a sugar donor, suggesting R699 is a mimiviral collagen galactosylhydroxylysyl glucosyltransferase (GGT). To facilitate further analysis of human and mimiviral homologous proteins, we presented an interactive and searchable genome-wide comparison website for quickly browsing human and Acanthamoeba polyphaga mimivirus homologs, which is available at RRID Resource ID: SCR_022140 or https://guolab.shinyapps.io/app-mimivirus-publication/.

H-Map-Based Technique for Mining High Average Utility Itemset

High Utility Itemset Mining (HUIM) is the process of locating itemsets that are profitable and useful to users. One of the key flaws in HUIM is that as the length of the itemset increases, the utility also increases. The true utility/profit of the itemset is not revealed in HUIM. High Average Utility Itemset mining overcomes the limitations of HUIM by taking the length of the itemset into account when estimating the utility. Existing pruning methods used for eliminating weak candidates overestimate the average usefulness of itemsets, causing the mining process to slow down. To prevent processing unpromising candidate itemsets and efficiently reduce the search space and processing time, the proposed methodology employs Upper Bound using Remaining Items Utility, Maximum Itemset Utility, and Sum of Maximum Utility in a Transaction. It also uses the multithreaded parallel approach to reduce the processing time. The H-Map-based data structure (H-Map) used for storing the utility values reduces the lookup time and joins used for itemset extension compared to existing state-of-the-art High Average Utility Itemset mining algorithms. The performance of the proposed work is evaluated in terms of memory usage and the time taken for processing. The proposed work increases the overall efficiency of the system employing effective pruning algorithms for pruning poor candidate itemsets and an efficient data structure for storing utility values.

An LSTM-based model for the compression of acoustic inventories for corpus-based text-to-speech synthesis systems

Large acoustic inventories must be used to produce speech close to natural quality. However, the concatenation cost space grows exponentially with the number of acoustic units in the acoustic inventory, increasing the latency of the unit selection algorithm, making algorithms unusable in real-time end-to-end systems. Even when data compression techniques are introduced, the model size is still high, representing a challenge for end-to-end systems. Thus, in this paper, we propose representing the concatenation cost space using LSTM (Long Short-Term Memory). The results show a 90% reduction in the size of the data space compared to all our previous techniques, and by an over 70% decrease in the look-up time. The proposed LSTM-based compression increases the responsiveness of the corpus-based text-to-speech systems significantly while keeping the overall speech quality at the same level.

An optimal content indexing approach for named data networking in software‐defined IoT system

The evolution of the Internet of Things (IoT) has increased the number of connected devices in the network. This has shifted the focus from IP-based network architecture towards content-centric networking (CCN). CCN eliminates the need for address-content binding in the conventional IP-based networks and allows the content to be accessed based on the name instead of the physical location. Named data networking (NDN) is a promising technique that can fulfil the increasing demand for connected devices through the CCN approach. NDN distributes the content on the network and focusses on the security of the content rather than the communication channel. However, the increase in traffic due to the escalation in the number of connected devices can lead to congestion in the network. The content distribution approach on the nodes is generalised and suitable for small networks. In the case of larger networks, an optimal approach is required to decide the optimal location to store the required content. However, a linear search approach is used to search (or lookup) the content in the assigned cache of the NDN node. In this work, the authors have combined the software-defined networking (SDN) with the NDN approach to overcome the above-highlighted challenge. Thus, the authors have designed an optimal content storage and indexing approach based on NDN-SDN coalesce in the IoT ecosystem. The proposed approach includes different phases, (a) a hashing-based content searching approach is formulated to reduce the look-up time of the content, (b) a red-black tree-based content storage approach is introduced for optimal utilisation of the assigned cache memory of the different NDN nodes, and (c) SDN controller facilitates automated network management and helps to administer the network requirements centrally and locate the content accordingly. The proposed approach was validated through the simulation experiments concerning network delay, packet rate, throughput, and cache hit ratio. The results obtained show the effectiveness of the proposed approach.

Electrical Automation Engineering Based on AI Technology

In the research of electrical automation engineering, AI technology has been widely used in industrial production and scientific research. It can help people do deeper and higher quality work, and also improve our ability to develop artificial intelligence systems. Therefore, this paper designs an electrical automation engineering system based on AI technology. Firstly, this paper introduces the concept and composition of electrical automation engineering, then expounds the definition, characteristics and classification of AI technology, then studies the existing problems of electrical automation engineering, designs the electrical automation engineering system framework based on AI technology, and tests the function of the system. Finally, the test results show that in terms of the stability of each module function of the system, the shortest stability time is at least 24 hours, which can ensure the stable operation of the system. In terms of the time spent connecting to the network, the DNS lookup time is within 3 seconds, the server processing time is within 2-3 seconds, and the content data transmission time is within 4 seconds. This shows that the system can run stably.

Flexible IP: An adaptable IP address structure and its efficient addressing scheme

The conventional IP address is designed with fixed length and lacking of extensibility, while the demand for addresses varies greatly in different scenarios. Flexible IP (FlexIP), as a variable length IP address, proactively makes address structure flexible enough to adapt to various network cases. Different lengths of the addresses could be used to accommodate different demands. However, how to efficiently addressing with length variable addresses is still a problem to be solved. The Bloom filter-based addressing scheme appears to be an excellent candidate with the possibility of compact storage and efficient member query. In this paper, we propose an adaptable IP Address Structure, which is expected to proactively adapt various network cases under flexible address structure and make Internet Protocol agilely cover futuristic and unknown scenarios. Moreover, we propose an OBF (One Bloom filter) based addressing scheme using only one Bloom filter for FlexIP. While keeping nearly the same false positive ratio as the conventional Bloom filter-based scheme, the OBF-based scheme significantly improves the addressing efficiency. OBF-based has two key features, one is that it achieves constant, yet small IP lookup time, and another is that it is insensitive to the length of the address. Simulation results show that the addressing scheme we proposed is more suitable for FlexIP addressing than well-known schemes.

GIF IMAGE HARDWARE COMPRESSORS

Increasing requirements for data transfer and storage is one of the crucial questions now. There are several ways of high-speed data transmission, but they meet limited requirements applied to their narrowly focused specific target. The data compression approach gives the solution to the problems of high-speed transfer and low-volume data storage. This paper is devoted to the compression of GIF images, using a modified LZW algorithm with a tree-based dictionary. It has led to a decrease in lookup time and an increase in the speed of data compression, and in turn, allows developing the method of constructing a hardware compression accelerator during the future research.

DynamicTuple: The dynamic adaptive tuple for high-performance packet classification

The frequent rule updating in Software Defined Networking and cloud computing requires the packet classification algorithm to perform fast packet matching and flow table updating simultaneously. The existing algorithms, e.g. decision-tree-based or hash-based, achieve fast packet classification at the sacrifice of the rule updating or vice versa. We propose the Dynamic Adaptive Tuple (DynamicTuple) for both fast packet classification and rule updating simultaneously. DynamicTuple builds a performance model of tuple based packet classification and exploits dynamic programming to find the appropriate tuple formulation to minimize the lookup time. When the rules keep updating, DynamicTuple can retain the proper tuples and rebuild the improper tuples, thus the tuple in DynamicTuple is dynamic adaptive. The experimental results demonstrate that DynamicTuple achieves 33.1x, 5.6x, 5.7x classification speed and 2.9x, 11.0x, 8.5x updating speed compared to Tuple Space Search, TupleMerge, PartitionSort. By implementing DynamicTuple, the OpenFlow table of Open vSwitch achieves 26.6x classification speed.

Industrial Networks Driven by SDN Technology for Dynamic Fast Resilience

Software-Defined Networking (SDN) provides the prospect of logically centralized management in industrial networks and simplified programming among devices. It also facilitates the reconfiguration of connectivity when there is a network element failure. This paper presents a new Industrial SDN (ISDN) resilience that addresses the gap between two types of resilience: the first is restoration while the second is protection. Using a restoration approach increases the recovery time proportionally to the number of affected flows contrarily to the protection approach which attains the fast recovery. Nevertheless, the protection approach utilizes more flow rules (flow entries) in the switch which in return increments the lookup time taken to discover an appropriate flow entry in the flow table. This can have a negative effect on the end-to-end delay before a failure occurs (in the normal situation). In order to balance both approaches, we propose a Mixed Fast Resilience (MFR) approach to ensure the fast recovery of the primary path without any impact on the end-to-end delay in the normal situation. In the MFR, the SDN controller establishes a new path after failure detection and this is based on flow rules stored in its memory through the dynamic hash table structure as the internal flow table. At that time, it transmits the flow rules to all switches across the appropriate secondary path simultaneously from the failure point to the destination switch. Moreover, these flow rules which correspond to secondary paths are cached in the hash table by considering the current minimum path weight. This strategy leads to reduction in the load at the SDN controller and the calculation time of a new working path. The MFR approach applies the dual primary by considering several metrics such as packet-loss probability, delay, and bandwidth which are the Quality of Service (QoS) requirements for many industrial applications. Thus, we have built a simulation network and conducted an experimental testbed. The results showed that our resilience approach reduces the failure recovery time as opposed to the restoration approaches and is more scalable than a protection approach. In the normal situation, the MFR approach reduces the lookup time and end-to-end delay than a protection approach. Furthermore, the proposed approach improves the performance by minimizing the packet loss even under failing links.

Updatable learned index with precise positions

Index plays an essential role in modern database engines to accelerate the query processing. The new paradigm of "learned index" has significantly changed the way of designing index structures in DBMS. The key insight is that indexes could be regarded as learned models that predict the position of a lookup key in the dataset. While such studies show promising results in both lookup time and index size, they cannot efficiently support update operations. Although recent studies have proposed some preliminary approaches to support update, they are at the cost of scarifying the lookup performance as they suffer from the overheads brought by imprecise predictions in the leaf nodes. In this paper, we propose LIPP, a brand new framework of learned index to address such issues. Similar with state-of-the-art learned index structures, LIPP is able to support all kinds of index operations, namely lookup query, range query, insert, delete, update and bulkload. Meanwhile, we overcome the limitations of previous studies by properly extending the tree structure when dealing with update operations so as to eliminate the deviation of location predicted by the models in the leaf nodes. Moreover, we further propose a dynamic adjustment strategy to ensure that the height of the tree index is tightly bounded and provide comprehensive theoretical analysis to illustrate it. We conduct an extensive set of experiments on several real-life and synthetic datasets. The results demonstrate that our method consistently outperforms state-of-the-art solutions, achieving by up to 4X for a broader class of workloads with different index operations.

Data Membership Identification using Bloom Filter in Cloud Storage for Effective Resource Allocation

Cloud computing has recently been the buzz word that had changed the entire software industry with its PaaS, IaaS and SaaS architecture model. The legacy systems operational in the organizations across different industries have been using the database that had an overhead in terms of cost of data storage, runtime operation and frequent data maintenance. Cloud database concept had challenged the existing storage and operational norms of data. In the distributed environment, resources used in the cloud databases need to identify whether the requested data belongs to the data nodes of a cluster. With databases began to be ubiquitous, the data storage needed to satisfy heterogeneous data structures rather the unstructured data storage support is looked for. The Use of Bloom's filter for data membership identification is the novel approach and can effectively improve the resource organization strategy on cloud. Dynamic resource organization can further improve the query efficiency as well. The concern raised during this is the data privacy which can also be ascertain by maintaining the data access authority levels. Bloom filter uses less memory space against the large dataset to store it's information. A Bloom filter is proposed to be used to determine whether an element is part of a reference set. It is a very compact hash-based data structure with efficient look-up times and a manageable risk of giving false positives.

Dictionary lookup with one genome evolution operation

SummaryGiven an m‐length query string q, approximate dictionary lookup searches for strings in a string dictionary D at a distance of 1 to q under some distances. In biological retrieval systems, the distances in such queries are defined by evolution operations on genomes. We consider the approximate dictionary lookup with one genome evolution operation including reversal and transposition, which searches for strings in D that can be generated from q by one reversal or one transposition. When the length of the reversed substring is confined to a constant α>1, we propose an O(m)‐time approach which uses bits space, where the dictionary D has d strings with totally |D| symbols. If the lengths of the reversals are in a range [α,β], the time for query is , and the space is words for any constant ε, in which occ is the number of matched strings. For problems allowing one transposition, when the length of the transposition is fixed to α, the time for a dictionary lookup is , while using words. In the case that the two swapped substrings are of the same length, the time for answering the query is O(m), while the space is bits.

An efficient pipeline processing scheme for programming Protocol-independent Packet Processors

OpenFlow is unable to provide customized flow tables, resulting in memory explosions and high switch retirement rates. This is the bottleneck for the development of SDN. Recently, P4 (Programming Protocol-independent Packet Processors) attracts much attentions from both academia and industry. It provides customized networking services by offering flow-level control. P4 can “produce” various forwarding tables according to packets. P4 increases the speed of custom ASICs. However, with the prevalence of P4, the multiple forwarding tables could explode when used in large scale networks. The explosion problem can slow down the lookup speed, which causes congestions and packet losses. In addition, the pipelined structure of forwarding tables brings additional processing delay. In this study, we will improve the lookup performance by optimizing the forwarding tables of P4. Intuitively, we will install the rules according to their popularity, i.e., the popular rules will appear earlier than others. Thus, the packets can hit the matched rule sooner. In this paper, we formalize the optimization problem, and prove that the problem is NP-hard. To solve the problem, we propose a heuristic algorithm called EPSP (Efficient Pipeline Processing Scheme for P4), which can largely reduce the lookup time while keeping the forwarding actions the same. Because running the optimization algorithm frequently brings additional processing burdens, wedesign an incremental update algorithm to alleviate this problem. To evaluate the proposed algorithms, we set up the simulation environments based on ns-3. The simulation results show that the algorithm greatly reduces both the lookup time and the number of memory accesses. The incremental algorithm largely reduces the processing burdens while the lookup time remains almost the same with the non-incremental algorithm. We also implemented a prototype using floodlight and mininet. The results show that our algorithm brings acceptable burder, and performs better than traditional algorithm.

Multiclass Membership Determination Integrating an ID-Bound Method with Bloom Filter

Membership determination with classification is an essential function for a variety of applications, such as network routing and packet inspection. We aim to constitute a mechanism capable of performing such a service with the considerations of efficiency and error ratio issues. The proposed method employs an array structure, in which each unit composes of multiple bits. The array houses a circular bound representation and a Bloom filter, where the former encodes the members and makes judgments of queries, while the latter acts as a secondary means for programming cases indeterministic by the former. Experimental results show that the error ratios were competitive and consistent with different levels of classification, as well as the memory space utilized. The lookup time decreased as the memory space increased. We depict specific characteristics for tackling the problem, including simplicity of data structure for encoding the members, avoidance of checking all classification IDs in the lookup process, and adoption of an auxiliary means for improving the error ratios. The ID-bound method is novel, and with the Bloom filter as the assistant, the proposed approach is robust in performances of both efficiency and error ratio.

Multilevel length-based classified index table for IP lookups and updates

To obtain a high-speed IP address lookup algorithm for wire-speed packet forwarding in Internet routers, a new data structure, called multilevel length-based classified index table, is proposed. This data structure can be divided into multilevel structures on the basis of prefix length distribution. Each level has a corresponding data structure and dynamic router-table operations. Experiments were performed using real-world routing databases, wherein the proposed data structure was compared with other structures by using the benchmark IPv4 and IPv6 prefix databases AS1221, AS4637, AS6447, AS1221*, and AS6447*. The average lookup time and the average update time of the proposed structure were superior to those of the other data structures; the memory requirement was lower than that of simple structures, such as prefix trees and priority tries.