Multiple Storage Devices Research Articles

SCIPIS: Scalable and concurrent persistent indexing and search in high-end computing systems

While it is now routine to search for data on a personal computer or discover data online, there is no such equivalent method for discovering data on large parallel and distributed file systems commonly deployed on HPC systems. In contrast to web search, which has to deal with a larger number of relatively small files, in HPC applications there is a need to also support efficient indexing of large files. We propose SCIPIS, an indexing and search framework, that can exploit the properties of modern high-end computing systems, with many-core architectures, multiple NUMA nodes and multiple NVMe storage devices. SCIPIS supports building and searching TFIDF persistent indexes, and can deliver orders of magnitude better performance than state-of-the-art approaches. We achieve scalability and performance of indexing by decomposing the indexing process into separate components that can be optimized independently, by building disk-friendly data structures in-memory that can be persisted in long sequential writes, and by avoiding communication between indexing threads that collaboratively build an index over a collection of large files. We evaluated SCIPIS with three types of datasets (logs, scientific data, and metadata), on systems with configurations up to 192-cores, 768 GiB of RAM, 8 NUMA nodes, and up to 16 NVMe drives, and achieved up to 29x better indexing while maintaining similar search latency when compared to Apache Lucene.

Effective role of vibrational annealing in enhancing room temperature physical properties of Co doped ZnO nanoparticles by using novel TVA technique

Well-annealed nanocrystalline semiconductor materials in pellet or thin film form, composed of small grains with a high surface-to-volume ratio, are considered as promising candidates for multiple device applications. However, no such method is available yet that can control the grain size of such samples during heat treatment. In this present work, a novel thermo-vibrational annealing (TVA) setup for processing nanocrystalline materials was designed and fabricated in-house. Here both thermal and mechanical vibrational energy were simultaneously applied to the sample during annealing, followed by vibrational dry quenching in the presence of an inert gas environment. Co-doped ZnO pellets were synthesized by chemical route pyrophoric method followed by annealing at 800 °C for 4 h using this set up. For comparison, one pellet was annealed with vibration with calibrated vibration at a speed of 360 rpm, and another pellet was annealed without vibration. X-ray diffraction (XRD), Raman and Fourier transform Infrared (FTIR) spectroscopic studies showed the formation of a single crystalline phase with hexagonal wurtzite structure of both the samples and the proper incorporation of Co2+ ions into the ZnO matrix without any impurity phases, which was also confirmed by energy dispersive X-ray (EDX) spectroscopy studies. Scanning electron microscopy (SEM) and high-resolution transmission electron microscopy (HRTEM) studies confirmed the decrease in average grain and particle size of the sample when processed with the TVA setup. Ultraviolet–Visible spectra showed that the bandgap energy of the Co-doped ZnO sample varies inversely with particle size. Hall-effect measurements demonstrated the semiconducting nature of both samples with n-type charge carriers. The presence of Co2+ ions in the wurtzite ZnO lattice was confirmed by X-ray photoelectron spectroscopy (XPS). The magnetic properties were studied using a SQUID magnetometer, which confirmed that a Co doped ZnO sample annealed with the TVA technique exhibit better room temperature ferromagnetism properties compared to sample annealed without vibration. Based on these studies, it was believed that the sample annealed by mechanical vibrational energy with vibrational dry quenching has a great influence on several physical properties of any material and hence Co doped ZnO sample processed with this TVA technique can be used as a promising candidate for multiple sensor and magnetic storage devices.

Optimal Energy Management and Storage Sizing for Electric Vehicles With Dual Storage

Battery degradation reduces the performance and lifetime of electric vehicles (EVs). Using energy storage devices with different characteristics alongside the battery can minimize degradation while satisfying driving demands. However, this introduces the additional complexities of sizing multiple storage devices and controlling them in real time. In this brief, we first provide a computationally tractable method to manage power-sharing between dual energy storages using approximate linear programming (ALP), an approximation of infinite horizon dynamic programming (DP). We formulate a procedure to determine the optimal sizes of the two storages based on the solution to the energy management problem to account for the tradeoff between vehicle range, storage size, and weight. We validate our approaches on a numerical case study. Numerical results show that our controller shares power efficiently between the storages, and our sizing procedure provides a design with minimal cost.

The State of the Art of Metadata Managements in Large-Scale Distributed File Systems — Scalability, Performance and Availability

File system metadata is the data in charge of maintaining namespace, permission semantics and location of file data blocks. Operations on the metadata can account for up to 80% of total file system operations. As such, the performance of metadata services significantly impacts the overall performance of file systems. A large-scale distributed file system (DFS) is a storage system that is composed of multiple storage devices spreading across different sites to accommodate data files, and in most cases, to provide users with location independent access interfaces. Large-scale DFSs have been widely deployed as a substrate to a plethora of computing systems, and thus their metadata management efficiency is crucial to a massive number of applications, especially with the advent of the Big Data age, which poses tremendous pressure on underlying storage systems. This paper reports the state-of-the-art research on metadata services in large-scale distributed file systems, which is conducted from three indicative perspectives that are always used to characterize DFSs: high-scalability, high-performance, and high-availability, with special focus on their respective major challenges as well as their developed mainstream technologies. Additionally, the paper also identifies and analyzes several existing problems in the research, which could be used as a reference for related studies.

Development of a Personalized Recommendation System for E-Commerce Products for Distributed Storage Systems.

Because the distributed storage system is based on network technology, it can store data in multiple independent low-cost physical storage devices, and it is also suitable for large-capacity storage, so it has become more and more popular. Today, common applications of distributed storage systems include cloud storage services, data center storage services, and P2P storage services. Typical ones are GFS, HDFS, OceanStore, and Dynamo. Due to regional and economic differences, the development level of global e-commerce (b2c) is very inconsistent. b2c contains the following key tags: buying and selling, which is the core of the website platform. E-commerce provides business users with transparent information and high-quality cheap products. Logistics is the basic guarantee for customers to execute transactions, and it is also a strict indicator of the website platform. There will be many visits during the operation of the e-commerce system, and the number of users in the early stage will increase exponentially. A safe and efficient e-commerce system can provide users with one-stop transaction support and convenient transaction processes. The personalized recommendation system has formulated some rules for certain fields, based on these rules, and defined certain types of knowledge for certain items to meet the needs of certain users and use the defined reasoning rules to generate recommendation results.

Joint Power and Subchannel Allocation for Distributed Storage in Cellular-D2D Underlays.

Wireless distributed storage is beneficial in the provision of reliable content storage and offloading of cellular traffic. In this paper, we consider a cellular device-to-device (D2D) underlay-based wireless distributed storage system, in which the minimum storage regenerating (MSR) coding combined with the partial downloading scheme is employed. To alleviate burdens on insufficient cellular resources and improve spectral efficiency in densely deployed networks, multiple storage devices can simultaneously use the same uplink cellular subchannel under the non-orthogonal multiple access (NOMA) protocol. Our objective is to minimize the total transmission power for content reconstruction, while guaranteeing the signal-to-interference-plus-noise ratio (SINR) constraints for cellular users by jointly optimizing power and subchannel allocation. To tackle the non-convex combinational program, we decouple the original problem into two subproblems and propose two low-complexity algorithms to efficiently solve them, followed by a joint optimization, implemented by alternately updating the solutions to each subproblem. The numerical results illustrate that our proposed algorithms are capable of performing an exhaustive search with lower computation complexity, and the NOMA-enhanced scheme provides more transmission opportunities for neighbor storage devices, thus significantly reducing the total power consumption.

Safe Operation Conditions of Electrical Power System Considering Power Balanceability among Power Generators, Loads, and Storage Devices

The introduction of an energy storage system plays a vital role in the integration of renewable energy by keeping a stable operation and enhancing the flexibility of the power flow system, especially for an islanding microgrid which is not tied to a grid and for a self-contained microgrid which tries to stay independent from a grid as much as possible. To accommodate the effects of power fluctuations of distributed energy resources and power loads on power systems, a power flow assignment under power balance constraint is essential. However, due to power limitations of power devices, the capacity of storage devices, and power flow connections, the power balance may not be achieved. In this paper, we proposed a system characterization which describes the relation among power generators, power loads, power storage devices, and connections that must be satisfied for a system to operate by keeping SOC limitations of power storage devices. When we consider one power generator, one power load, and one power storage device connected at a single node, the generated energy by the generator minus the consumed energy by the load from some start time will increase/decrease the state of charge (SOC) for the storage device; hence, keeping SOC max/min limitations relies on whether the difference between the generated energy and the consumed energy stays within a certain range or not, which can be computed from the capacity Ess and other parameters. Our contribution in this paper is an extension and generalization of this observation to a system that consists of multiple fluctuating power generators, multiple fluctuating power loads, multiple storage devices, and connections that may not be a full connection between all devices. By carefully enumerating the connection-dependent flow paths of generated energy along the flow direction from generators to storages and loads, and enumerating the connection-dependent flow paths of consumed energy along the counter-flow direction from loads to storages and generators, we have formulated the increase/decrease of SOCs of storage devices caused by the imbalance between generated energy and consumed energy. Finally, considering the max/min limitations of SOCs and fluctuations of power generators and power loads, the conditions that the power generators and the power loads must have for SOCs of storage devices to maintain individual max/min limitations have been derived. The system characterization provides guidelines for a power flow system that can continue safe operation in the presence of power fluctuations. That is, in order for a system to have a feasible power flow assignment, there are the issues of how large the capacity of a power storage device should be, how large/small the maximum/minimum power/demand levels of the power generators and the power loads should be, and how the connection should be configured. Several examples using our system characterization are demonstrated to show the possible applications of our results.

Coordination Control and Suitability Evaluation of Multiple Ice-Storage Tanks in an Air-conditioning System Under Unreliable Communication

Electric energy is an important part of building energy consumption, and air-conditioning accounted for most of the consumption of electric energy. Ice storage air conditioning is widely concerned because of its strong adjustability. Due to the limited space, it is often necessary to use multiple ice storage devices simultaneously to meet the application requirements. In this paper, a distributed coordination method for multiple ice storage devices is presented, which can allocate storage tasks according to the capacity of the device. Each device only needs to communicate with its neighbouring cold storage device, and finally achieve the control goal through distributed iteration. The convergence of the proposed algorithm is analysed by rigorous mathematics. If the communication topology between devices is connected, the algorithm can converge to the expected point. Finally, the validity and suitability of the algorithm and the correctness of theoretical analysis are verified and evaluation by digital simulation.

Design and implementation of an I/O isolation scheme for key-value store on multiple solid-state drives

High-performance storage devices, such as Non-Volatile Memory express Solid-State Drives (NVMe SSDs), have been widely adopted in data centers. Especially, multiple storage devices provide higher I/O performance compared with a single device. However, the performance can be reduced in the case of workloads with mixed read and write requests (e.g., key-value stores) even though multiple storage devices are adopted. This is because read requests can be blocked until the processing for write requests is finished. In this article, we propose an I/O isolation scheme to improve the performance of the key-value store for multiple SSDs. In our scheme, we classify files of the key-value store and deploy files to the separated storage devices according to the characteristics of each file. Thus, read/write operations are performed in different storage devices. In addition, we propose two different device mapping methods, namely fixed and adaptive device mapping to deploy files to the proper device. We implement our scheme in RocksDB with multiple storage devices (six NVMe SSDs) and extend our scheme on an open-channel SSD, which reveals internal hardware architecture to verify the effectiveness of read/write isolation within a single storage device. The experimental results demonstrate that our scheme improves performance by up to 29% and 26% in the open-channel SSD and multiple storage devices, respectively, compared with the existing scheme.

Full-temperature covered switching material with triple optic-dielectric states in a lead-free hybrid perovskite

Optic-electric responsive materials have attracted much attention for their applications in temperature-sensing, actuators, and memory switches. However, it is a challenge to integrate various functions to form multifunctional responsive materials. As molecule-based hybrid materials usually consist of organic and inorganic components, the introduction of multiple functions can be achieved through structural construction. Thus far, even though full-temperature cover is required for device applications, full-temperature covered multi-switchable hybrid materials have rarely been successfully synthesized. Herein, the dynamic [(CH3)3NOH]+ cation and luminous center Mn(II) were introduced to form a hybrid material [(CH3)3NOH][MnCl3], showing multiple temperature-responsive behaviors. Upon temperature change, it exhibits multi-state dielectric switching response and intensity or peak shift response of luminous in full-temperature range (low, room, and high temperatures). These responsive behaviors are triggered by the motion or reorientation of [(CH3)3NOH]+ cations and inorganic framework. Overall, the switchable photoelectric material has potential applications in multiple encrypted storage and sensor devices.

A redistribution tool for long-term archive of astronomical observation data

Astronomical observation data require long-term preservation, and the rapid accumulation of observation data makes it necessary to consider the cost of long-term archive storage. In addition to low-speed disk-based online storage, optical disk or tape-based offline storage can be used to save costs. However, for astronomical research that requires historical data (particularly time-domain astronomy), the performance and energy consumption of data-accessing techniques cause problems because the requested data (which are organized according to observation time) may be located across multiple storage devices. In this study, we design and develop a tool referred to as AstroLayout to redistribute the observation data using spatial aggregation. The core algorithm uses graph partitioning to generate an optimized data placement according to the original observation data statistics and the target storage system. For the given observation data, AstroLayout can copy the long-term archive in the target storage system in accordance with this placement. An efficiency evaluation shows that AstroLayout can reduce the number of devices activated when responding to data-access requests in time-domain astronomy research. In addition to improving the performance of data-accessing techniques, AstroLayout can also reduce the storage systems power consumption. For enhanced adaptability, it supports storage systems of any media, including optical disks, tapes, and hard disks.

A Benchmark Distribution System for Investigation of Residential Microgrids With Multiple Local Generation and Storage Devices

A benchmark distribution system is developed for investigating control and energy management of distributed generation (DG) at a residential level in the form of three single-phase microgrids. The benchmark is derived from a typical distribution network architecture with common parameters found in North-America systems including wiring specifications, line impedances and connection details for rooftop PV systems. This benchmark system can accommodate microgrids operating in both grid-connected and islanded modes. Within this benchmark, multiple single-phase DG sources located in different phases can be coordinated to form a dynamically balanced three phase system under different load and generation profiles in different phases. The coordination of DG sources in a particular phase is achieved through an intra-phase power management device, while mitigating loads and generation imbalance among all phases are done by an inter-phase power management scheme. It is expected that this benchmark system will facilitate investigation of impacts posed by proliferation of single-phase distributed generation devices and local storage systems in private residences. Three case studies have been carried out to demonstrate the versatility and effectiveness of this benchmark system.

CSMqGraph: Coarse-Grained and Multi-external-storage Multi-queue I/O Management for Graph Computing

As graphs continue growing, external storage graph processing systems serve as a promising alternative to distributed in-memory solutions for low cost and high scalability. To obtain high I/O throughput, these systems usually use multiple external storage devices. They adopt the operating system I/O management method based on striped volume, resulting in unsatisfactory performance, such as low sequential bandwidth utilization of each external storage device, limited I/O parallelism and expensive management overhead. In this paper, we analyzed the problems of the operating system I/O management method based on striped volume. Then we designed CSMqGraph, a graph processing system adopts coarse-grained striping method matching sequential large I/O to fully utilize the maximum sequential bandwidth of each external storage device and an I/O management strategy based on multi-external-storage multi-queue making I/O threads dedicated to each external storage device to further improve I/O throughput and fully exploit the parallelism of multiple external storage devices. For different graph algorithms and datasets, our evaluation shows that CSMqGraph consistently outperforms state-of-the-art engines GridGraph by up to 40%, and has better I/O scalability.

키밸류 스토어의 파일 I/O 특성을 고려한 다중 저장장치로의 효율적 파일 배치 기법

최근 많은 데이터 센터들에서는 나날이 증가하는 데이터를 관리하기 위해 고성능 저장장치를 채택하고 있다. 또한, 다수의 장치를 사용하여 대용량 데이터의 효율적으로 처리한다. 그러나, 고성능 저장장치를 사용함에도 불구하고 읽기와 쓰기 연산이 혼재되어 발생하는 일부 응용에서는 성능이 저하될 수 있다. 이는 장치 내부에서 쓰기 연산의 처리가 완료될 때까지 읽기 연산의 처리가 지연되기 때문이다. 본 논문에서는 응용에서 생성하는 파일들의 I/O 특성을 고려하여 다수의 장치에 효율적으로 분리 저장하는 기법을 제안한다. 실험 결과를 통해 읽기와 쓰기 연산을 분리하여 처리함으로써 응용의 성능이 최대 21% 향상됨을 보인다.

A survey on access control mechanisms for cloud computing

AbstractCloud computing is an Internet‐based computing where the information technology resources are provided to end users following their request. With this technology, users and businesses can access programs, storage, and application development platforms through the Internet and via the services offered by the cloud service providers (CSPs). One of the biggest obstructions in the cloud computing environment is data security. Actually, the data are dispersed across multiple machines and storage devices such as servers, computers, and various mobile devices. The uncontrolled access to these resources and data leads to many important data security risks for the end users. In this way, and in order to ensure the reliability of the cloud and the trust of the users regarding this environment, controlling access to data and resources as well as protecting and ensuring their security becomes a critical task for CSPs. In this work, we present a comprehensive review of existing access control mechanisms used in the cloud computing environment. The advantages and disadvantages of each of these models are discussed and presented along with their analysis. Also, we study the cloud requirements of these models, and we evaluate existing control mechanisms against these requirements.

Defense Against Advanced Persistent Threats in Dynamic Cloud Storage: A Colonel Blotto Game Approach

Advanced persistent threat (APT) attackers apply multiple sophisticated methods to continuously and stealthily steal information from the targeted cloud storage systems and can even induce the storage system to apply a specific defense strategy and attack it accordingly. In this paper, the interactions between an APT attacker and a defender allocating their central processing units (CPUs) over multiple storage devices in a cloud storage system are formulated as a Colonel Blotto game. The Nash equilibria of the CPU allocation game are derived for both symmetric and asymmetric CPUs between the APT attacker and the defender to evaluate how the limited CPU resources, the data storage size and the number of storage devices impact the expected data protection level and the utility of the cloud storage system. A CPU allocation scheme based on “hotbooting” policy hill-climbing that exploits the experiences in similar scenarios to initialize the quality values to accelerate the learning speed is proposed for the defender to achieve the optimal APT defense performance in the dynamic game without being aware of the APT attack model and the data storage model. A hotbooting deep ${Q}$ -network-based CPU allocation scheme further improves the APT detection performance for the case with a large number of CPUs and storage devices. Simulation results show that our proposed reinforcement learning-based CPU allocation can improve both the data protection level and the utility of the cloud storage system compared with the ${Q}$ -learning-based CPU allocation against APTs.

Integrity and Privacy through Authentication Key Exchange Protocols for Distributed Systems

Networking is the practice of connecting several computing devices together in order to share resources.In real world, attacks via force and fraud are privacy (unauthorized release of information), Integrity (tampering with data), Service (denial of service).The goals are disallow unauthorized access, allow authorized access, resist DOS attacks .In recent years, many efficient AKE protocols have been proposed to achieve user privacy and integrity in the communications. A communication model is a representation where there are a large number of clients accessing multiple remote and distributed storage devices in parallel. Authenticated key exchange (AKE) protocol allows a user and a server to authenticate each other and generate a session key for the later communications. This paper focuses on how to exchange key materials and establish parallel secure sessions between the clients and the storage devices in the Network in an efficient and scalable manner.

A Layered Methodology for the Simulation of Extra-Functional Properties in Smart Systems

Smart systems represent a broad class of intelligent, miniaturized devices incorporating functionality like sensing, actuation, and control. In order to support these functions, they must include sophisticated and heterogeneous components, such as sensors and actuators, multiple power sources and storage devices, digital signal processing, and wireless connectivity. The high degree of heterogeneity typical of smart systems has a heavy impact on their design: the challenges are not in fact restricted to their functionality, but are also related to a number of extra-functional properties, including power consumption, temperature, and aging. Current simulation- or model-based design approaches do not target a smart system as a whole, but rather single domains (digital, analog, power devices, etc.) or properties. This paper tries to overcome this limitation by proposing a framework for the concurrent simulation of both functionality and such extra-functional properties. The latter are modeled as different information flows, managed by dedicated “virtual buses” and formalized through the adoption of IP-XACT. SystemC, through the support of physical and continuous time modeling provided by its analog and mixed signal extension, is used to implement both functional and extra-functional models. Experimental results show the efficiency, accuracy and modularity of the proposed approach on an example case study, in which substantial speedups with respect to standard model-based design tools go along with a very high degree of accuracy (<; 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-5</sup> %). Furthermore, the case study highlights that the proposed framework allows to easily capture at run time the mutual impact of properties, e.g., in case of power and temperature.

Parallel network file systems using authenticated key exchange protocols

The keyestablishment for secure many-to-many communications is very important nowadays. The problem is inspired by the proliferation of large-scale distributed file systems supporting parallel access to multiple storage devices. In this, a variety of authenticated key exchange protocols that are designed to address the issues. This shows that these protocols are capable of reducing the workload of the metadata server and concurrently supporting forward secrecy and escrow-freeness. All this requires only a small fraction of increased computation overhead at the client. This proposed three authenticated key exchange protocols for parallel network file system (pNFS). The protocols offer three appealing advantages over the existing Kerberos-based protocol. First, the metadata server executing these protocols has much lower workload than that of the Kerberos-based approach. Second, two of these protocols provide forward secrecy: one is partially forward secure (with respect to multiple sessions within a time period), while the other is fully forward secure (with respect to a session). Third, designed a protocol which not only provides forward secrecy, but is also escrow-free.

Evolutionary Game Theoretic Analysis of Advanced Persistent Threats Against Cloud Storage

Advanced Persistent Threats (APTs) represent stealthy, powerful, long-term, and well-funded attacks against cyber systems, such as data centers and cloud storage. Evolutionary game theory is used to capture the long-term continuous behavior of the APTs on the cloud storage devices. Two APT defense games with discrete strategies are formulated, in which both an APT attacker and a defender compete to control one or multiple storage devices regarding their attack or defense intervals. The dynamical stability of each defense and attack strategy pair is studied according to the replicator dynamics criteria to characterize the locally asymptotically stable equilibrium strategies. The evolutionary stable strategy is discussed in each game, which is a subset of the asymptotically stable Nash equilibrium (NE). The phase portraits provide the locally asymptotically stable points of the APT defense game, which represent the NE showing the relationship between the asymptotic stability and evolutionary stability.