Distributed Storage Framework Research Articles

Volatility-based diversity awareness for distributed data storage of Mobile Crowd Sensing

With the widespread utilization of sensors and the rapid development of Mobile Crowd Sensing (MCS), the framework of distributed data storage, which makes use of the limited storage capacity of sensing devices, is providing an alternative to cloud-based data storage. Since the construction of the distributed storage framework for MCS has to be completed by Compressed Sensing (CS) theory, a reasonable measurement allocation becomes a no-brainer if we want to obtain more recovery precision of data without wasting measurement resources. To more efficiently address the above problems, we present a novel strategy of Volatility-Based Diversity Awareness for CS measurement allocation called VBDA, which collectively achieves lower computational complexity, adaptive sampling rate allocation for non-visual data, and high-quality reconstruction. We first process the target monitoring region in blocks. Then, we calculate the degree of data diversity of each area using the volatility, which is used to assess the importance of the different areas. Worthy note, based on the volatility calculation, we can obtain the magnitude of the variation in diversity that is only present in the real data, even when using very ambiguous recovered data. This is primarily due to the volatility unique design concept, which attempts to offset partially identical errors in adjacent recovered data using volatility calculations. Finally, to rationally allocate measurement resources, a volatility-based sampling rate allocation scheme is proposed. We further provide an implementation for practical deployments of VBDA in various related contexts. Numerous experimental show that the VBDA performs excellently. In comparison to the current state-of-the-art strategy, it improves data recovery accuracy by 12.2% without the use of any prior knowledge, while being adaptable to many different distribution types of data.

SECURE AND EXPRESSIVE DATA ACCESS CONTROL FOR CLOUD STORAGE, BY USING CRYPTCLOUD+

An emerging form of cloud administration, secure distributed storage looks to protect sensitive data while still making it easily accessible to users regardless of where their data resides. A promising method that could be used to confirm the administration’s credibility is Encryption Using Ciphertext-Policy Attributes (CP- ABE). However, due to the “win or bust” decoding feature of CP-ABE, its use may result in an inevitable security breach known as the abuse of access certification (for example, unscrambling rights). On the one hand, side of the semi-believed specialist, the other is in, and favour of the an online client; both are studied in this paper. We propose CryptCloud+, a CP-ABE-based distributed storage framework with white-box recognizability and reviewing, to help curb abuse. We also detail the security analysis and provide experimental evidence of our framework value. INDEX TERMS: Attribute-based Ciphertext-Policy Encryption (CP- ABE), Inevitable Security Breach, win or bust, curb abuse

MapChain-D: A Distributed Blockchain for IIoT Data Storage and Communications

With the rapid growth of the Industrial Internet of Things (IIoT) devices, managing extensive volume of IIoT data becomes a significant challenge. While the conventional cloud storage approaches with centralised data centres suffer from high latency for large-scale IIoT data storage due to the increased communications and latency overheads, distributed storage frameworks such as blockchains have become promising solutions. In this paper, we design and analyse a dual-blockchain framework for secure and scalable distributed data management in large-scale IIoT networks. The proposed framework, named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> , consists of a data chain that is mapped to an index chain to provide efficient data storage and lookup. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> is designed for practical IIoT applications with storage, latency, and communications constraints. Detailed data exchange protocols are presented for the data insertion and retrieval operations in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> . Based on these, theoretical analyses are provided on the space, time, and communications complexities of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> compared with conventional single-chain frameworks with local and distributed data storage. We implement our <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> prototype using open-source LoRaWAN communications with multiple RPi and Arduino devices, Kademlia-based distributed hash table (DHT), and Ethereum-based blockchain with proof-of-authority (PoA) consensus. Experimental results from our prototype show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> is more suitable to be deployed on resource-constrained IIoT devices. We also highlight the scalability and flexibility of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MapChain-D</i> with different numbers of edge nodes in the system.

Origami Store: UC-Secure Foldable Datachains for the Quantum Era

Traditional blockchains preserve everything, if verification is to avoid trusted third parties. The accumulation of outdated but undeletable data makes traditional blockchains non-scalable for practical applications. Stateless blockchains mitigate this problem via history deletion; however, existing stateless blockchains such as Mimblewimble are not quantum resistant as they rely on specific discrete-log-based cryptography to achieve history deletion. In this paper, we introduce the “Origami Store” (or O-store for short), as a general new approach to distributed data storage with a generic history deletion mechanism called “folding.” O-store is built using a hash function and an arbitrary signature scheme. Since the signature scheme is decoupled from the framework, O-stores are compatible with any quantum-resistant signature scheme by design. More importantly, O-store only needs to preserve one signature and one verification key per account (or owner) for any number of transactions (or updates), thereby mitigating the inefficiency of large quantum-safe signatures and verification keys. We prove the security of the basic O-store using the framework of Universal Composability (UC); in particular, O-store's universal security is shown to hold for a generic consensus mechanism, to facilitate implementation from a wide range of decentralized consensus protocols. The Origami store is thus the first UC-secure efficient distributed data storage framework that supports plug-and-play consensus mechanisms for fully decentralized trustless operation. We demonstrate working implementations of Origami stores with three post-quantum signature libraries-qTESLA, Dilithium, and Picnic-to illustrate its scalability and practicality for real-world applications.

Forestry big data platform by Knowledge Graph

Using the advantages of web crawlers in data collection and distributed storage technologies, we accessed to a wealth of forestry-related data. Combined with the mature big data technology at its present stage, Hadoop’s distributed system was selected to solve the storage problem of massive forestry big data and the memory-based Spark computing framework to realize real-time and fast processing of data. The forestry data contains a wealth of information, and mining this information is of great significance for guiding the development of forestry. We conducts co-word and cluster analyses on the keywords of forestry data, extracts the rules hidden in the data, analyzes the research hotspots more accurately, grasps the evolution trend of subject topics, and plays an important role in promoting the research and development of subject areas. The co-word analysis and clustering algorithm have important practical significance for the topic structure, research hotspot or development trend in the field of forestry research. Distributed storage framework and parallel computing have greatly improved the performance of data mining algorithms. Therefore, the forestry big data mining system by big data technology has important practical significance for promoting the development of intelligent forestry.

The Method Balance Consistency Availability on Systems Virtual Machine

In this time, a few associations are putting away their information on distributed storage so as to meet the prerequisites of effective activity like soundness, versatility, and accessibility of administrations. Information replication benefits in distributed storage frameworks are there to improve execution. In this unique situation, the prerequisites for guaranteeing information consistency have turned out to be progressively significant. In this paper, we have proposed a virtual server solution for refreshing replication, effectively settling assets, distinguishing and avoiding gridlocks at the server sides. In this original study, we have demonstrated that the proposed arrangement yields the outcomes for the construction which guarantees information consistency in regard to expenses and idleness. we utilize the Open Stack apparatus, which joins a proposed calculation on Adjusting Consistency Accessibility in Physical Machine for keeping up information consistency in Distributed storage Frameworks.

A proficient remote information responsibility check protocol in multi-cloud environment

Remote information respectability is one of the testing issues in the distributed storage framework which guarantees to deal with the re-appropriating information securely without downloading the entire information. Because of some useful circumstances, there is a need to spare the customer’s information is distributed over multi-cloud servers. In the interim, the information honesty confirmation convention ought to viable for diminishing the verifier cost. Existing information respectability approaches depend on hash capacities and advanced marks which are not fitting for the remote multi-cloud condition because of high computational over-burden and remote in nature. Furthermore, another current issue is that aggressors can hack the information in few occasions from the cloud regardless of the way that information respectability checking convention was presented. Remembering these issues, in this work, we anticipate a multi-cloud confined information respectability checking convention. The proposed technique offers consistent assertion towards the customer and assesses the data, which is immovably settled with the help of the new information respectability checking convention. At the point when an aggressor attempts to take the data in multi-cloud, the proposed continuous reviewing system will help the verifier to execute square dimension checking to ensure the information respectability through the one-round test-response or test reaction technique. A multi-cloud data integrity verification protocol is presented in test plan that depends on signature and symmetric keys. Using the proposed method, the clients can test the engaged information in one phase test reaction interaction with little transmission cost. In addition, it empowers public auditing and helps dynamic information preservation that enables the clients to adjust and erase the data with minimized execution overhead. For evaluating the execution of the proposed technique interms of information trustworthiness, a broad arrangement of experimentation happens. The proposed work enables various clients to manage remote cloud information. The execution of the proposed method is verified concerning update, verify, and inquiry time cost. From the accomplished outcomes, it is discovered that the proposed technique ensures the nature of administration and fixed security by having reasonable refresh time and a strict confirmation rule separate.

Implementation of key-cryptosystem for efficient data sharing in cloud environment

The distributed storage based on data recovery administration is a promising innovative way that mounts market actively sooner rather than later. However, in spite of the various studies which have been done in terms of secure information recovery over encoded information in cloud administrations, the majority of these studies concentrated on the strict security given to information in an outside space. Those methodologies oblige dynamite expenses to unify on cloud administration supplier that can be an important hindrance to accomplish productive information recovery in distributed storage. In this paper, a proficient information recovery plan utilizing property based encryption was proposed. The proposed plan is most appropriate for distributed storage frameworks with the gigantic measure of infor-mation. Affluent expressiveness as access control and quick pursuits with straightforward examinations of looking elements were given. Additionally, the plan ensures information security and client protection among information recovery process.

Secured information exchange in cloud using cross breed property based encryption

Individuals have the ability to get to the web anyplace and whenever. Distributed computing is an idea that treats the assets on the Internet as a brought together element, to be specific cloud. The server farm administrators virtualized the assets as per the necessities of the clients and uncover them as capacity pools, which the clients can themselves use to store documents or information protest s. Physically, the assets may get put away over numerous servers. Thus, Data heartiness is a noteworthy prerequisite for such stockpiling frameworks. In this paper we have proposed one approach to give information power that is by imitating the message with the end goal that every capacity server stores a duplicate of the message. We have improved the safe distributed storage framework by utilizing a limit intermediary re-encryption strategy. This encryption conspire bolsters decentralized deletion codes connected over scrambled messages and information sending operations over encoded and encoded messages. Our framework is exceedingly dispersed where every capacity server freely encodes and forward messages and key servers autonomously perform incomplete decoding.

Extensible and fine-grained characteristics-positioned information storage in cloud computing

With the improvement of distributed computing, outsourcing information to cloud server pulls in loads of considerations. To ensure the security and accomplish adaptable fine-grained record access control, (ABE) was proposed and utilized as a part of distributed storage framework. Be that as it may,

Generalized Pyramid Codes for Versioned Data in Distributed Storage Networks

A distributed storage framework called sparsity exploiting coding (SEC) was recently proposed for storing multiple versions of data object, wherein the objects are stored in a differential manner in order to reduce the I/O reads when retrieving multiple versions. It was shown that the design of erasure codes for the SEC framework requires Cauchy-matrix-based maximum distance separable (MDS) codes, which, however, do not enjoy the locally repairable property during the repair process. In this letter, we study the suitability of generalized pyramid codes (GPCs), a well-known class of locally repairable codes, to the SEC framework, and analyze their capability to retrieve multiple versions with few I/O reads. We show that GPCs can retrieve correlated multiple versions with fewer I/O reads than the overall object size, however, requiring more I/O reads than that by Cauchy-matrix-based MDS codes. This penalty in the I/O reads comes at the advantage of availing the locally repairable property in the SEC framework. We also show that GPCs require fewer I/O reads than the optimal locally repairable codes, in some cases.

Time Series Data Storage System Based on Big Data Technologies: Architecture and Implementation

AbstractA steady usage growth of various smart meters/sensors types comes along with the development of Smart Grid. Due to that smart meters/sensors usually produce data continuously, the state-of-the-art storage architecture will no longer be sustainable under such a dramatic data increase circumstance. It is crucial that these data shall be stored for both short and long term analysis to seek valuable information. This paper proposed a scalable and distributed storage architecture for time series data, and further substantiated the proposed storage framework on a proof-of-concept testbed using a cluster of low-cost and credit-card-sized single-board computers. We evaluated the proposed distributed storage framework and proof-of-concept testbed with a comprehensive set of performance measures. Results showed that low-cost hardware and open source technologies could be utilized for large-scale time series data storage.

A Secured Cloud Data Storage with Access Privilages

In proposed framework client source information reinforcements off-site to outsider distributed storage benefits to decrease information administration costs. In any case, client must get protection ensure for the outsourced information, which is currently safeguarded by outsiders. A configuration and instrument FADE, and a safe overlay distributed storage framework that achieve fine-grained, strategy based methodology control and document guaranteed erasure. It partners outsourced records with document association approaches, and without a doubt erases records to make them unrecoverable to endless supply of document access arrangements, To accomplish such security objectives, FADE is based upon an arrangement of cryptographic key operations that are self-kept up by a majority of key supervisors that are free of outsider mists. In unmistakable, FADE goes about as an overlay framework that works flawlessly on today's distributed storage administrations. Actualize a proof-of-idea model of FADE on Amazon S3, one of today's distributed storage administrations. By behavior broad true studies, and confirm that FADE gives security insurance to source information, while presenting just insignificant behavior and financial cost overhead. My work oversee, esteem included security highlights acclimatize were today's distributed storage administration.

Analytic Queries over Geospatial Time-Series Data Using Distributed Hash Tables

As remote sensing equipment and networked observational devices continue to proliferate, their corresponding data volumes have surpassed the storage and processing capabilities of commodity computing hardware. This trend has led to the development of distributed storage frameworks that incrementally scale out by assimilating resources as necessary. While challenging in its own right, storing and managing voluminous datasets is only the precursor to a broader field of research: extracting insights, relationships, and models from the underlying datasets. The focus of this study is twofold: exploratory and predictive analytics over voluminous, multidimensional datasets in a distributed environment. Both of these types of analysis represent a higher-level abstraction over standard query semantics; rather than indexing every discrete value for subsequent retrieval, our framework autonomously learns the relationships and interactions between dimensions in the dataset and makes the information readily available to users. This functionality includes statistical synopses, correlation analysis, hypothesis testing, probabilistic structures, and predictive models that not only enable the discovery of nuanced relationships between dimensions, but also allow future events and trends to be predicted. The algorithms presented in this work were evaluated empirically on a real-world geospatial time-series dataset in a production environment, and are broadly applicable across other storage frameworks.

Autonomous Cloud Federation for High-Throughput Queries over Voluminous Datasets

The breadth and depth of information being generated and stored continues to grow rapidly, causing an information explosion. Observational devices and remote sensing equipment are no exception here, giving researchers new avenues for detecting and predicting phenomena at a global scale. To cope with increasing storage loads, hybrid clouds offer an elastic solution that also satisfies processing and budgetary needs. In this article, the authors describe their algorithms and system design for dealing with voluminous datasets in a hybrid cloud setting. Their distributed storage framework autonomously tunes in-memory data structures and query parameters to ensure efficient retrievals and minimize resource consumption. To circumvent processing hotspots, they predict changes in incoming traffic and federate their query resolution structures to the public cloud for processing. They demonstrate their framework's efficacy on a real-world, petabyte dataset consisting of more than 20 billion files.

Unified Data Access Security on RevocableMulti Authority CP-ABE in Cloud

The cloud information administrations, it is typical for information to be put away in the cloud, as well as imparted crosswise over various Users. A few systems have been intended to permit both information holders and open verifiers to proficiently review cloud information trustworthiness without recovering the whole information from the cloud server. Be that as it may, open examining on the respectability of imparted information to these current systems will definitely uncover classified data character protection to open verifiers. Information access control is a successful approach to guarantee the information security in the cloud. Because of information outsourcing and untrusted cloud servers, the information access control turns into a testing issue in distributed storage frameworks. Our property renouncement technique can effectively attain to both forward security and retrogressive security.

분산 클러스터 메모리 기반 대용량 OWL Horst Lite 온톨로지 추론 기법

Current ontology studies use the Hadoop distributed storage framework to perform map-reduce algorithm-based reasoning for scalable ontologies. In this paper, however, we propose a novel approach for scalable Web Ontology Language (OWL) Horst Lite ontology reasoning, based on distributed cluster memories. Rule-based reasoning, which is frequently used for scalable ontologies, iteratively executes triple-format ontology rules, until the inferred data no longer exists. Therefore, when the scalable ontology reasoning is performed on computer hard drives, the ontology reasoner suffers from performance limitations. In order to overcome this drawback, we propose an approach that loads the ontologies into distributed cluster memories, using Spark (a memory-based distributed computing framework), which executes the ontology reasoning. In order to implement an appropriate OWL Horst Lite ontology reasoning system on Spark, our method divides the scalable ontologies into blocks, loads each block into the cluster nodes, and subsequently handles the data in the distributed memories. We used the Lehigh University Benchmark, which is used to evaluate ontology inference and search speed, to experimentally evaluate the methods suggested in this paper, which we applied to LUBM8000 (1.1 billion triples, 155 gigabytes). When compared with WebPIE, a representative mapreduce algorithm-based scalable ontology reasoner, the proposed approach showed a throughput improvement of 320% (62k/s) over WebPIE (19k/s).

A distributed storage framework of FlowTable in software defined network

Openflow, a novel Software Defined Network (SDN) technology, is developing rapidly and has already been utilized in many fields. It facilitates decoupling between the control and forwarding plane, enabling users to code the network functions easily and replace the traditional high-cost network functions devices. The FlowTable of Openflow, its base of operating the network packets, consists of many flow entries and is stored in the Ternary Content Addressable Memories (TCAMs) of the Openflow switch. When the FlowTable occupies the entire storage space of the Openflow switch and more flow entries are added, the delete operation on the TCAMs increases, and the latency and loss of packets deteriorates – this is the primary issue. In order to solve this problem, this paper proposes a distributed storage framework which stores the FlowTable in multiple Openflow switches, equipped with small TCAMs. To conclude, this paper simulates the algorithms used in the framework and builds a testbed. The experimental results prove the framework’s feasibility and successful performance.

Design and Implement Enhancing Security in Multi Cloud Storage System using Distributed File System

Distributed computing is a quickest developing engineering. It permits business associations to utilize or access diverse applications, store data without access their individual documents. While considering the force, strength and the security of cloud one can't disregard diverse dangers to client's information on distributed storage. Document access guarantee in true procedure to the record assurance because of entrust cloud servers. In distributed storage framework document doorway system is all the more difficult issue. This framework in result produces repetitive duplicates of comparative documents or includes a totally dependable cloud server. Assaults from enemy client are hard to stop in distributed storage. In proposed framework we are actualizing the idea of various distributed storage alongside improved security utilizing encryption strategies where rather putting away finish record on single cloud framework. The framework will part the record in distinctive lumps then encode it and store on diverse cloud. The information needed for unscrambling and revamping that document will be put away in metadata administration server for effective recovery of unique record.