Data Retrieval Operations Research Articles

Secure Distributed Storage Orchestration on Heterogeneous Cloud-Edge Infrastructures

Distributed storage systems spanning across different cloud data centers have substantially improved availability and flexibility for data storage and retrieval operations. However, stringent latency requirements of emerging applications necessitate optimized selection of storage resources that exhibit smaller delay. Introducing edge resources into distributed storage systems enables data placement closer to its source, but simultaneously increases the complexity of decision-making and orchestration processes for optimal data placement. In this work, we develop mechanisms for storing data across an infrastructure that includes both edge and cloud resources. Our approach focuses on optimizing data integrity, longevity, security, and cost, while leveraging erasure coding when performing the resource allocation. We first present a comprehensive mixed integer linear programming formulation of the storage resource orchestration problem. As the search space for the optimal solution can be vast and the execution time prohibitively large for real size problems, we also propose an innovative multi-agent heuristic approach that uses the rollout, a reinforcement based policy, to balance performance and execution time efficiently. Through various simulation experiments, we evaluate the developed mechanisms and trade-offs involved in our approach. By incorporating data from a multi-cloud provider, we further enhance the validity of the simulations and the conclusions drawn.

MapChain-D: A Distributed Blockchain for IIoT Data Storage and Communications

With the rapid growth of the Industrial Internet of Things (IIoT) devices, managing extensive volume of IIoT data becomes a significant challenge. While the conventional cloud storage approaches with centralised data centres suffer from high latency for large-scale IIoT data storage due to the increased communications and latency overheads, distributed storage frameworks such as blockchains have become promising solutions. In this paper, we design and analyse a dual-blockchain framework for secure and scalable distributed data management in large-scale IIoT networks. The proposed framework, named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> , consists of a data chain that is mapped to an index chain to provide efficient data storage and lookup. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> is designed for practical IIoT applications with storage, latency, and communications constraints. Detailed data exchange protocols are presented for the data insertion and retrieval operations in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> . Based on these, theoretical analyses are provided on the space, time, and communications complexities of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> compared with conventional single-chain frameworks with local and distributed data storage. We implement our <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> prototype using open-source LoRaWAN communications with multiple RPi and Arduino devices, Kademlia-based distributed hash table (DHT), and Ethereum-based blockchain with proof-of-authority (PoA) consensus. Experimental results from our prototype show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> is more suitable to be deployed on resource-constrained IIoT devices. We also highlight the scalability and flexibility of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> with different numbers of edge nodes in the system.

An efficient and secure data sharing scheme for cloud data using hash based quadraplet wavelet permuted cryptography approach

SummaryEnsuring the reliable and secure data transmission in cloud systems is very essential and demanding task in the recent decades. For that purpose, various security frameworks have been deployed in the existing works, which are mainly focusing the improving the privacy and confidentiality of cloud data against the unauthenticated users. Yet, it facing the major problems of inefficient accessing, increased time consumption, computational complexity, storage complexity, and memory utilization rate. Hence, this research work intends to develop an advanced and efficient cryptography model, named as, hash based quadraplet wavelet permuted cryptography (HQWPC) for the secured cloud data storage and retrieval operations. Here, the bilinear mapping and group hash function generation processes are performed to generate the keys used for cryptographic operations. Also, the zig‐zag scanning, wavelet coefficients extraction, and permutation processes are accomplished for data encryption. Consequently, the inverse of these operations is performed while decrypting the data. In addition to that, an integrated signature control policy authentication mechanism is employed for validating the authenticity of the cloud data users. This kind of signature verification process could efficiently increase the security level of cloud data. For validating the performance of the proposed security framework, various evaluation metrics have been utilized during analysis. Then, the obtained results are compared with the recent state‐of‐the‐art models for proving the efficiency of the proposed technique over the other techniques.

A survey on Vehicular Fog Computing: Current state-of-the-art and future directions

Vehicular fog computing (VFC) enhances Intelligent Traffic System (ITS) by computing the real-time traffic information for accident alerts, path navigation and etc. It utilizes parked and moving vehicles as a fog node to perform computation rather than the core cloud. VFC consumes less bandwidth, decreased response time, and congestion of core cloud due to sort distance. VFC performs resource allocation to select the appropriate vehicle for computation according to the available computational resource. It performs data retrieval operations to transfer traffic data from vehicle to vehicular fog because direct communication is limited by the transmission range of the vehicle. VFC faces various issues during resource allocation and data retrieval operation such as vehicle's high mobility, dynamic topology and etc. This causes uneven resource distribution due to inefficient resource allocation and late data delivery due to unfeasible data retrieval. The security and privacy leakage of traffic data threaten due to sharing in vehicular fog. To overcome these issues, we provide in-depth exploration at each phase of the VFC operation. Hence, we present a new classification of VFC operation, which classifies the challenges every phase such as resource allocation in task division, scheduling, and load balancing; result-retrieval in phase, node selector, node selection and path recovery; secure data sharing in authentication, encipherment, auditing, and data privacy. This state-of-the-art gives a better understanding of open research issues and future direction to efficiently handle the computation at VFC.

KVSTL: An Application Support to LSM-Tree Based Key-Value Store via Shingled Translation Layer Data Management

LSM-tree based Key-value (KV) stores greatly fit the needs of write-intensive applications with its efficient data store and retrieval operations to datasets. To accommodate ever-growing datasets, shingled magnetic recording (SMR) drives have become a popular option to provide large storage capacity for KV stores at low cost. SMR drives achieve high storage density via overlapping tracks on the disk surface. However, the overlapped track layout induces the sequential-write constraint and prevents KV stores from storing and rearranging KV pairs efficiently. In this paper, we present KVSTL, a KV store aware Shingled Translation Layer (STL), to preserve the merits of existing KV stores, while exploiting the high storage density of SMR drives. KVSTL is proposed as an application support to hide the management complexity of SMR drives and facilitates the management of SMR drives via passing only the “level” and “invalidation” information of LSM-tree based KV stores onto SMR drives. The proposed KVSTL achieves its performance enhancement via managing key-value pairs with level awareness and enabling efficient storage space management with the invalidation information. The results show that KVSTL can reduce the written data amount for 69.45 percent on average and the latency for up to 62.72 percent when compared with SMR-based LevelDB.

WITHDRAWN: Improving electoral roll authenticity and accessibility using side-chaining and deep learning

Abstract The main issue for any organisational elections is the validity and accessibility of the electoral roll. The electoral roll requires data validation when data is added or updated, and requires effective data selection when data retrieval operations are required. It is an evolving process that needs continuous moderation from both the bureaucrats and the public. Because of their inherent properties, such as non-mutability, multi-level authentication, cryptography, and strict data update laws, Blockchain driven systems have proven to be efficient in terms of security. But due to large data chains, blockchain systems have limited retrieval performance, which slows down the read-cycles, rendering the system ineffective for big data applications. A side-chain-based solution for the efficient storage and recovery of electoral roll data is described in this paper. In comparison to single-chain systems, data updating and insertion into side chains is less reliable. This is due to the fact that as opposed to their single chain counterparts, side chains are very limited in scale, so targeting these side chains is simpler. In order to preserve the security of the side-chain scheme, this paper describes a deep learning cryptosystem (DLC). In order to create a special non-identifiable side chain structure, this DLC utilises a combination of adaptive cryptography and transfer learning. In order to ensure high fidelity and reduce the risk of attacks, this side-chain system uses a hybrid blockchain implementation. The proposed method, while retaining the same degree of accuracy and protection, is found to be 47 percent faster than the current single-chain electoral roll management solutions.

Towards the design of efficient hash-based indexing scheme for growing databases on non-volatile memory

The index is a fundamental component in data intensive systems to accelerate data retrieval operations. In the design of Non-Volatile Memory (NVM) based indexes, the hash-based structure is one of the most promising candidates since it can take full advantages of byte-addressable property of NVM to perform query operations with constant time complexity. However, we found that the basic operation, “rehash operation”, may incur a large number of write activities on NVM, which is harmful to the endurance of NVM, and will cause drastic performance degradation. Additionally, range query operations cannot be efficiently conducted on hash-based indexes. In this paper, we first investigate how to design an NVM-friendly hash-based structure with the considerations of endurance and performance issues. Then, we propose a novel indexing scheme called “Bucket Hash”, which can significantly reduce the overhead caused by rehash operations and range query operations. We evaluate the proposed Bucket Hash using YCSB workloads. Compared with existing indexes, Bucket Hash achieves 40% reduction on average in the number of NVM writes, meanwhile gaining 30% improvement on timing performance.

Towards the Design of Efficient and Consistent Index Structure with Minimal Write Activities for Non-Volatile Memory

Index structures can significantly accelerate the data retrieval operations in data intensive systems, such as databases. Tree structures, such as B $^{+}$ -tree alike, are commonly employed as index structures; however, we found that the tree structure may not be appropriate for Non-Volatile Memory (NVM) in terms of the requirements for high-performance and high-endurance. This paper studies what is the best index structure for NVM-based systems and how to design such index structures. The design of an NVM-friendly index structure faces a lot of challenges. First , in order to prolong the lifetime of NVM, the write activities on NVM should be minimized. To this end, the index structure should be as simple as possible. The index proposed in this paper is based on the simplest data structure, i.e., linked list. Second , the simple structure brings challenges to achieve high-performance data retrieval operations. To overcome this challenge, we design a novel technique by explicitly building up a contiguous virtual address space on the linked list, such that efficient search algorithms can be performed. Third , we need to carefully consider data consistency issues in NVM-based systems, because the order of memory writes may be changed and the data content in NVM may be inconsistent due to write-back effects of CPU cache. This paper devises a novel indexing scheme, called “ V irtual L inear A ddressable B uckets” (VLAB). We implement VLAB in a storage engine and plug it into MySQL. Evaluations are conducted on an NVDIMM workstation using YCSB workloads and real-world traces. Results show that write activities of the state-of-the-art indexes are 6.98 times more than ours; meanwhile, VLAB achieves 2.53 times speedup.

An Efficient and Scalable RDF Indexing Strategy based on B-Hashed-Bitmap Algorithm using CUDA

ABSTRACT Indexing enormous databases such as RDF has been a focus of intense research. As is well understood, indexing plays a pivotal role in speeding up data retrieval operations and query performance. Besides expediting search, an index can motivate new data-store schemes and technologies that can possibly revolutionize large data-analytics engine design, more often relevant to semantic web. Due to the proliferation of internet and the ease of creating and generating data on the fly - handling, storing and the subsequent semantic processing has proven to be a major bottleneck for the RDF data community. Handling data of such scale and magnitude requires a parallel approach as provided by the GPUs (Graphical processing units). In this paper, a new efficient and scalable index is proposed that uses a combination of B+ trees, hashing and sparse matrices. These data structures have an edge over others in terms of their implementation as a parallel algorithm using the CUDA (Compute Unified Device Architecture) framework meant to program massively parallel GPU multicores. So far, RDF data has been mostly implemented either as a RDBMS or as a non-native data-store, in both cases the sequential indexing strategy fails miserably with the scaling of the data-store. Parallel implementation of indices provides a suitable option for dealing with scalable and dynamically generated data over distributed networks. The crucial sparse matrix part of the proposed index is benchmarked against different CUDA memory implementations to derive optimal matrix processing options. The sparse matrix search is profiled using cudamemchk and visual profiler for identifying bottlenecks and inconsistencies in thread execution called thread divergence. Benchmarking the data provides promising results for a B+ tree based index coupled with hashing and sparse matrix implementations.

RIOS: efficient I/O in reverse direction

SummaryThe reverse mode of automatic differentiation executes the adjoint statements induced by each statement in the original program in the reverse order of the original program flow. This program flow reversal commonly requires storage of information on the control flow of the original program. In addition, intermediate values of variables that are overwritten have to be recorded, as these values may later be needed to compute the partial derivatives of the corresponding statement. The stored information will be accessed in reverse order of being written. This runs contrary to many assumptions made in standard implementations of file systems, operating systems, and input/output (I/O) libraries. A common buffering strategy aimed at speeding up future read requests is to employ read‐ahead. This strategy is useful for accesses in forward direction but is considered to be harmful to the performance of the reverse mode. To increase the performance of the reverse mode, it is also advantageous to interleave computations with the data storage and retrieval operations, which can be achieved using multithreading. To this end, we design and implement a novel software called reverse‐mode I/O stream (RIOS) that is adapted to these particular requirements of the reverse mode. We show the advantages of RIOS in two empirical case studies, an artificially constructed example of a typical I/O pattern in the reverse mode and a real‐world example arising from fluid mechanics, which is studied in Fortran90 and in Matlab where the reverse mode is generated via the automatic differentiation tools Tapenade and ADiMat, respectively. Copyright © 2014 John Wiley &amp; Sons, Ltd.

Obtaining the Dewey Decimal Classification Number from other databases: a catalog clean-up project

The increasing availability of online catalogs and bibliographical databases allows not only for copy cataloging, but also for the retrieval of atomic information useful within the catalog. To this end, Dewey decimal numbers were imported from national and international sources by means of the unique identifier ISBN. Technical specifications have been developed to locate the records to be retrieved, to query external databases, to extract the Dewey decimal classification numbers and add them to the catalog. The exceptionally large amount of Dewey numbers added to the catalog has improved the semantic usability of the OPAC. The procedure established has also facilitated the collection of information on the use of the Dewey Decimal System in the various databases used and allowed to make certain comparisons between them. The tools employed can be used analogously for data-retrieval operations in the catalog, as an aid in the cataloging process, or to improve the OPAC in either a static or dynamic manner. Taking into account its virtually exclusive practical purpose, this work is characterized by practical rather than theoretical choices. However, the experience acquired opens up areas even in the field of academic research.

Genre-specific modeling of visual features for efficient content based video shot classification and retrieval

This paper presents a genre-specific modeling strategy capable of improving the task of content based video classification and the speed of data retrieval operations. With the ever increasing growth of video data it is important to classify video shots into groups based on its content. For that reason, it is of primary concern to design systems that could automatically classify videos into different genres based on its content. We consider the genre recognition task as a classification problem. We use support vector machines to perform the classification task and propose an improved video classification method. The experimental results show that genre-specific modeling of features can significantly improve the performance. Results have been compared with two contemporary works on video classification, to demonstrate the superiority of our proposed framework.

XQuery as a retrieval mechanism for longitudinal multiscale forest resource data

In most forested environments, boreal forest in particular, monitoring and evaluation of management activities would be facilitated by storage of multiscale data sets spanning long time periods. For this study, such a data repository based on a generic spatio-temporal conceptual data model was created using XML. The data repository containing changes in data content for a 1000 ha forest estate between 1871 and 1994 was used to evaluate the data retrieval capabilities of XQuery. The data retrieval operations consisted of pivoting, sequencing, and aggregation. Pivoting transforms the generic attribute storage used in the data model into a more compact column-wise format. Sequencing is used to return a temporal sequence of values, and for the traversal of the object hierarchy. Aggregation aggregates the values of a variable over a grouping variable. The combinations of the basic retrieval operations were also investigated as well as the effect of time and object hierarchy on the retrieval operations. The conceptual match between the generic spatio-temporal data model and the XML data model was apparent at the data retrieval operations. The key properties of XQuery for the implementation of the retrieval operations were extendability, support for nested structures, and data transformation functionality.

DNA databases

This paper presents DNA algorithms for five relational algebra database operations, selection, projection, union, set difference, and Cartesian product on so-called DNA databases. A DNA database is a database where data records are encoded as DNA strands. The five operations mentioned before are fundamental in the field of databases and perform most of the data retrieval operations on current databases.

Dynamic cell-based MAC protocol for target detection applications in energy-constrained wireless networks

Today's advances in sensor technology make it feasible to embed sensing, communication and computation capabilities in small untethered nodes. However, node lifetime is still severely restricted by the limitations of power supply. By improving power consumption efficiency of sensor node operations, a sensor node's lifetime can be significantly extended. A well designed data gathering protocol can achieve this goal by minimizing the amount of data transmitted through the network. We believe that in scenarios where data retrieval operations are infrequent and localized, pre-configuring an entire sensor network is detrimental to power conservation. We propose and evaluate a new data gathering protocol, and demonstrate the advantages of a dynamically configured network for specific types of applications.

Combining Partitions in SecSpaces

SecSpaces is a data-driven coordination model that supports a tuple-based coordination space extended with mechanisms for controlling and authenticating the access to entries. This is achieved exploiting the notion of (symmetric and asymmetric) abstract partitions inside the space. In this paper we consider one of the limitations of SecSpaces: it is not well suited for supporting the atomic access to more than one partition at once. In order to tackle this limitation we extend the SecSpaces model introducing an operator to combine partitions; output operations can use the new operator to make an entry visible in more than one partition, and data-retrieval operations can use it to access atomically more than one partition. We formally define, in terms of a process calculus, this notion and we demonstrate the flexibility of this new mechanism via examples.

Improving the I/O performance of intermediate multimedia storage nodes

The data retrieval operations in servers and proxies for Media-on-Demand applications represent a severe bottleneck, because a potentially (very) high number of users concurrently retrieve data with high data rates. In the Intermediate Storage Node Concept (INSTANCE) project, we have developed a new architecture for Media-on-Demand storage nodes that maximizes the number of concurrent clients a single node can support. We avoid the traditional bottlenecks, like copy operations, multiple copies of the same data element in main memory, and checksum calculation in communication protocols, by designing, implementing, and tightly integrating three orthogonal techniques: a zero-copy-one-copy memory architecture, network level framing, and integrated error management. In this paper, we describe the INSTANCE storage node, and present an evaluation of our mechanisms. Our experimental performance results show that the integration of these three techniques in NetBSD at least doubles the number of concurrent clients that a single storage node can serve in our testbed.

Experience with SAND-Tcl: A Scripting Tool for Spatial Databases

The use of scripting makes it possible to overcome many important difficulties in the development of database applications. By extending a general-purpose scripting language with constructs derived both from the database kernel and from the intended application domain, issues such as query processing and user interfacing can be approached in an economical and flexible way. This is illustrated by describing our experience with SAND-Tcl, a scripting tool developed by us for building spatial database applications. SAND-Tcl is an extension of the Tcl-embedded scripting language with the constructs of the SAND environment for developing applications involving both spatial and non-spatial data. SAND-Tcl acts as a ‘glue’ to hold together all the subsystems of SAND. In fact, query evaluation plans are SAND-Tcl programs (or scripts) which are written on-the-fly by SAND in response to a query defined by the user. This permits the rapid prototyping of algorithms and makes SAND a useful tool both for applications and research. The focus is on data storage, retrieval operations, and spatial indexing. Implementations of operations such as spatial selection, ranking and spatial join are given. In addition, tools are described to make possible the construction of graphical user interfaces to a spatial database as well as providing users the ability to view and interact with spatial objects in a graphical manner. This is achieved through the use of SAND-Tcl scripts and the Tk graphical user interface toolkit which is tightly coupled to Tcl.

Efficient data retrieval and manipulation using Boolean entity lattice

This paper proposes a technique to organize the database such that retrievals and manipulations will be operated on a subset instead of the whole entity set. The conceptual partitioning lattice and the Boolean partitioning lattice of an entity set are introduced as basic concepts. A technique is presented for determining all descendant atoms (subsets) of a given entity set. It is demonstrated that using the Boolean lattice to describe the structure between entity subsets is better than using an ordinary lattice. It is also demonstrated that using the Boolean lattice to organize a database can make data manipulation and retrieval operations more efficient.

A data-driven model for a subset of logic programming

There is a direct correspondence between semantic networks and a subset of logic programs, restricted only to binary predicates. The advantage of the latter is that it can describe not only the nodes and arcs comprising a semantic net, but also the data-retrieval operations applied to such nets. The main objective of this paper is to present a data-driven model of computation that permits this subset of logic programs to be executed on a highly parallel computer architecture. We demonstrate how logic programs may be converted into collections of data-flow graphs in which resolution is viewed as a process of finding matches between certain graph templates and portions of the data-flow graphs. This graph fitting process is carried out by messages propagating asynchronously through the data-flow graph; thus computation is entirely data driven, without the need for any centralized control and centralized memory. This permits a potentially large number of independent processing elements to cooperate in solving a given query.