Distributed Storage Nodes Research Articles

Meta-Block: Exploiting Cross-Layer and Direct Storage Access for Decentralized Blockchain Storage Systems

Decentralized storage systems such as blockchain storage applications adopt the distributed storage technology and use distributed storage nodes to store the persistent data. For each off-chain storage node, key-value (KV) stores are normally used to manage data. As the most common data structure for KV store, Log Structured Merge Tree (LSM-Tree) eliminates random write operations and keeps acceptable read performance. Although LSM-Tree-based decentralized storage system can provide a secure and reliable storage platform, the unique feature of blockchain applications is not fully exploited. In blockchain storage applications, the generation of keys for KV stores is based on the encrypted data, and the key determines the allocation of data. Since the granularity for a read/write request at the level of blockchain storage platform is much smaller than that at the level of LSM-Tree or flash memory, a physical block in a solid-state drive (SSD) could be filled with data from different system users. This mixture of workloads will lead to the inefficient usage of physical spaces in the SSD and cause extra compaction operations for LSM-Tree. This paper presents <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Meta-Block</i> , a cross-layer and efficient storage management strategy for decentralized blockchain storage applications. Meta-Block utilizes rich functionalities provided by the system infrastructure of open-channel SSD to provide direct storage accesses for the off-chain storage node. The objective is to capture the features of blockchain storage applications and reduce unnecessary read and write operations across different storage layers. As a cross-layer design, Meta-Block redesigns the organization of LSM-Tree, which can effectively reduce the write amplification. We also design a data prefetching strategy to speed up the indexing and enable direct storage access. We demonstrate the viability of the proposed technique using a set of extensive experiments. Experimental results show that Meta-Block can effectively reduce the write amplification and extend the lifetime of SSDs in comparison with representative schemes.

Optimal Storage Allocation for Delay Sensitivity Data in Electric Vehicle Network

For significant characteristics such as high efficiency and environmental protection, electric vehicles (EVs) have become a new technology trend in the intelligent transportation system (ITS). Data like real time traffic situation and charge point occupation in ITS shows significant real-time characteristics. Distributed storage system, an effective technology to ensure reliable sharing of dynamic data, is first adopted to storage allocation for delay-sensitive data in this article. In order to improve the recovery probability of delay-sensitive data within its timeliness, we establish an access queuing delay model based on the characteristics of sensitive data. Then, we find the optimal storage allocation strategy across distributed storage nodes for data with different delay threshold in terms of the recovery probability. The analysis shows that for data with low real-time performance and delay sensitivity, the maximal symmetric allocation is more excellent. For real-time data with low delay threshold, the minimal allocation is better under the condition of limited storage budget.

Secure Data Storage and Recovery in Industrial Blockchain Network Environments

The massive redundant data storage and communication in network 4.0 environments have issues of low integrity, high cost, and easy tampering. To address these issues, in this article, a secure data storage and recovery scheme in the blockchain-based network is proposed by improving the decentration, tampering-proof, real-time monitoring, and management of storage systems, as such design supports the dynamic storage, fast repair, and update of distributed data in the data storage system of industrial nodes. A local regenerative code technology is used to repair and store data between failed nodes while ensuring the privacy of user data. That is, as the data stored are found to be damaged, multiple local repair groups constructed by vector code can simultaneously yet efficiently repair multiple distributed data storage nodes. Based on the unique chain storage structure, such as data consensus mechanism and smart contract, the storage structure of blockchain distributed coding not only quickly repair the nearby local regenerative codes in the blockchain but also reduce the resource overhead in the data storage process of industrial nodes. Experimental results show that the proposed scheme improves the repair rate of multinode data by 9% and data storage rate increased by 8.6%, indicating to be promising with good security and real-time performance.

A Distributed Storage and Computation k-Nearest Neighbor Algorithm Based Cloud-Edge Computing for Cyber-Physical-Social Systems

The k-nearest neighbor (kNN) algorithm is a classic supervised machine learning algorithm. It is widely used in cyber-physical-social systems (CPSS) to analyze and mine data. However, in practical CPSS applications, the standard linear kNN algorithm struggles to efficiently process massive data sets. This paper proposes a distributed storage and computation k-nearest neighbor (D-kNN) algorithm. The D-kNN algorithm has the following advantages: First, the concept of k-nearest neighbor boundaries is proposed and the k-nearest neighbor search within the k-nearest neighbors boundaries can effectively reduce the time complexity of kNN. Second, based on the k-neighbor boundary, massive data sets beyond the main storage space are stored on distributed storage nodes. Third, the algorithm performs k-nearest neighbor searching efficiently by performing distributed calculations at each storage node. Finally, a series of experiments were performed to verify the effectiveness of the D-kNN algorithm. The experimental results show that the D-kNN algorithm based on distributed storage and calculation effectively improves the operation efficiency of k-nearest neighbor search. The algorithm can be easily and flexibly deployed in a cloud-edge computing environment to process massive data sets in CPSS.

DHC: A Distributed Hierarchical Clustering Algorithm for Large Datasets

Hierarchical clustering is a classical method to provide a hierarchical representation for the purpose of data analysis. However, in practical applications, it is difficult to deal with massive datasets due to their high computation complexity. To overcome this challenge, this paper presents a novel distributed storage and computation hierarchical clustering algorithm, which has a lower time complexity than the standard hierarchical clustering algorithms. Our proposed approach is suitable for hierarchical clustering on massive datasets, which has the following advantages. First, the algorithm is able to store massive dataset exceeding the main memory space by using distributed storage nodes. Second, the algorithm is able to efficiently process nearest neighbor searching along parallel lines by using distributed computation at each node. Extensive experiments are carried out to validate the effectiveness of the DHC algorithm. Experimental results demonstrate that the algorithm is 10 times faster than the standard hierarchical clustering algorithm, which is an effective and flexible distributed algorithm of hierarchical clustering for massive datasets.

Storage and Bandwidth Optimized Reliable Distributed Data Allocation Algorithm

Distributed storage allocation problems are an important optimization problem in reliable distributed storage, which aims to minimize storage cost while maximizing error recovery probability by optimal storage of data in distributed storage nodes. A key characteristic of distributed storage is that data is stored in remote servers across a network. Thus, network resources especially communication links are an expensive and non-trivial resource which should be optimized as well. In this article, the authors present a simulation-based study of the network characteristics of a distributed storage network in the light of several allocation patterns. By varying the allocation patterns, the authors have demonstrated the interdependence between network bandwidth, defined in terms of link capacity and allocation pattern using network throughput as a metric. Motivated by observing the importance of network resource as an important cost metric, the authors have formalized an optimization problem that jointly minimizes both the storage cost and the cost of network resources. A hybrid meta heuristic algorithm is employed that solves this optimization problem by allocating data in a distributed storage system. Experimental results validate the efficacy of the algorithm.

Optimal Storage Allocation for Wireless Cloud Caching Systems With a Limited Sum Storage Capacity

In wireless cloud storage systems, the recovery failure probability depends on not only wireless channel conditions but also storage size of each distributed storage node. For an efficient utilization of limited storage capacity and the performance characterization of allocation strategies, we asymptotically analyze the recovery failure probability of a wireless cloud storage system with a sum storage capacity constraint for both high SNR regime and low SNR regime. Then, we find the optimal storage allocation strategy across distributed storage nodes in terms of the asymptotic recovery failure probability. Our analysis reveals that the maximal symmetric allocation is optimal for high SNR regime and the minimal allocation (with $\lfloor T\rfloor$ complete storage nodes and an incomplete storage node) is optimal for low SNR regime, where $T$ is the sum storage capacity. Based on the numerical investigation, we also show that in intermediate SNR regime, a balance allocation between the minimal allocation and the maximal symmetric allocation would not be required if we select one between them according to SNR.

A Billboard Manager Based Model That Offers Dual Features Supporting Cloud Operating System And Managing Cloud Data Storage

With an increasing focus on mobility and an innate desire to access data and information efficiently and as quickly as possible, users are turning to technologies that can service their needs with the minimum of fuss and a high degree of reliability. In this paper, we have designed a proposed model that seeks to combine Billboard Manager with an open source cloud operating system known as eyeOS that, together, can help achieve the goals desired in a private cloud environment. The eyeOS is essentially a web-based operating system that allows file read/write/update operations in cloud. The architecture that we have proposed in this paper, involves a mix of the Internet, a private cloud zone, and a Billboard Manager managing an eyeOS powered layer. In this model, any user would be able to create, update, access, and delete almost any file from anywhere where there is Internet connectivity options available. The Billboard Manager would enable users to store or retrieve files across distributed storage nodes in a private cloud zone. The architecture that we have proposed is aimed at providing a thin-client architecture to any user with a PC, laptop, or a suitably-enabled mobile computing device, such that the user is able to collaborate and communicate his/her work within a private cloud zone from almost any location.

Construction of one special minimum storage regenerating code when α=2

Regenerating codes is one class of erasure codes for distributed storage. A [n, k, d, α, β] regenerating code can recover the original message from any k out of n distributed storage nodes and repair any failed node from other arbitrary d nodes. α is the number of symbols stored in one node and β is the number of symbols downloaded by a newcomer node. Minimum storage regenerating (MSR) code is a regenerating code attaining the minimal storage requirement. In this paper, we address the design of a special MSR code where α = 2 and the first fragment of a node stores original symbol (we call it hybrid systematic MSR code or HS-MSR code). We point out that there exists no exact-repair construction when k ⩾ 5. A simple exact linear construction is given when k = 2. Furthermore, by relaxing the condition of connecting any d nodes, we investigate the properties of quasi-cyclic regenerating code and propose a unified construction such that when k ⩾ 3, repairing process can be achieved with less repair bandwidth by repair-by-transfer form.

Economic incentives in information- centric networking: implications for protocol design and public policy

In this article, we build a simple engineering-economic model to evaluate the incentives of various network players to deploy distributed storage nodes (content stores and caches) to support information-centric network (ICN) architectures and the implications of those incentives on protocol design, industry structure and other policy issues, such as competition and network- neutrality. We find that without some explicit monetary compensation from publishers, networks will fail to deploy the socially optimal number of caches. We also study the social welfare implications of different cache deployment scenarios and identify two deployment scenarios that maximize social welfare. Finally, we show that ICN architectures provide numerous opportunities for large eyeball networks to leverage their terminating access monopoly to extract more profits from other network players. Hence, regulators must take steps to address issues such as interconnection and the role of caching infrastructure in differential quality of service provisioning in an ICN-based network architecture, in order to ensure socially desirable outcomes from their deployment.

Social Welfare Implications of Different Cache Deployment Scenarios

Content delivery from distributed storage nodes within the network provides both an evolutionary and a revolutionary means to deal with the proliferation of content retrieval on the Internet. Regardless of whether the architecture redirects requests to nearby nodes that possess copies of a desired object or directly searches for the desired object by name, distributed storage plays a vital role in facilitating content delivery. Despite the important role played by distributed storage nodes in the current and future Internet, it is not yet obvious who should deploy them. Further, the social welfare implications of different deployment scenarios are not well understood.In this paper, we use a simple economic model to study the welfare implications of different cache deployment scenarios. We show that the level of caching supplied with transparent caching is inefficient and results in a deadweight loss. With a payment flow from publishers for content delivery, the level of caching supplied increases because caching service providers can extract some of the publisher surplus for themselves.Moreover, we show two deployment scenarios that maximize social welfare when a payment flow exists between the publisher and the caching service provider. In the first, eyeball networks provide the caching infrastructure through a federation or a third-party entity that serves as a transaction broker between publishers and various networks. In the second, eyeball networks pay third-party content delivery networks an amount, equivalent to the realized bandwidth and transit savings from caching, to deploy caches. Even though both deployment scenarios maximize welfare, we show that the latter deployment scenario is unlikely to be realized in practice because eyeball networks do not share in the resulting value created. This explains the emergence of caching federations and content distribution networks (CDNs) becoming brokers for eyeball network caching services.

Dispatching design for storage-centric wireless sensor networks

In a large-scale storage-centric wireless sensor network (SWSN) where data from different clusters are archived at distributed storage nodes, the dispatching design problem is to determine one or multiple storage nodes for each cluster. To achieve high data fidelity in SWSNs, the dispatching design should aim to reduce the data loss due to the network congestion and, at the same time, to prolong the network lifetime by avoiding sending excessive traffic to some particular storage nodes in order to achieve energy consumption balance among the relaying sensors around the storage node. In this paper, we propose an h-peak dispatching design for SWSN. Under such a design, regular traffic volume from each cluster will receive guaranteed data fidelity, and over-expectation traffic will receive best-effort data fidelity. We use an h-peak model to characterize the traffic deviation which assumes that at most h clusters may have over-expectation traffic simultaneously at a storage node. By incorporating h into the dispatching decision rather than assuming that all clusters may reach their traffic spikes simultaneously, the proposed h-peak dispatching design can achieve high data fidelity in SWSNs.