High Data Availability Research Articles

Xel: A cloud-agnostic data platform for the design-driven building of high-availability data science services

This paper presents Xel, a cloud-agnostic data platform for the design-driven building of high-availability data science services as a support tool for data-driven decision-making. We designed and implemented Xel based on four main components: (a) a high level and driven-design framework for end-users to select analytic and machine learning tools from a service mesh and coupling them into the form of processing pipelines; (b) a new recursive ETL processing model to automatically convert the pipeline designs into infrastructure-agnostic software structures, which are deployed on multiple infrastructures; (c) an orchestration model for transparently managing the data delivery throughout each stage of the processing pipelines used in data science systems; and (d) a data decentralized model to transparently mask service unavailability such as cloud outages and unavailability of either applications or data. Real users created, by means of Xel, data science services such as deep learning analysis of scientific publications, clustering of movie reviews, and a cancer exploratory study. These services were evaluated as case studies that revealed the efficacy of this platform design for enabling end-users to create multiple types of data science pipelines without programming nor making configurations, and automatically masking unavailability of cloud resources and data. This platform is currently used to create a national cancer observatory and big data systems for fusing suicide, mental health, drug consumption and macroeconomic datasets to find spatiotemporal patterns.

BAuth-ZKP-A Blockchain-Based Multi-Factor Authentication Mechanism for Securing Smart Cities.

The overwhelming popularity of technology-based solutions and innovations to address day-to-day processes has significantly contributed to the emergence of smart cities. where millions of interconnected devices and sensors generate and share huge volumes of data. The easy and high availability of rich personal and public data generated in these digitalized and automated ecosystems renders smart cities vulnerable to intrinsic and extrinsic security breaches. Today, with fast-developing technologies, the classical username and password approaches are no longer adequate to secure valuable data and information from cyberattacks. Multi-factor authentication (MFA) can provide an effective solution to minimize the security challenges associated with legacy single-factor authentication systems (both online and offline). This paper identifies and discusses the role and need of MFA for securing the smart city ecosystem. The paper begins by describing the notion of smart cities and the associated security threats and privacy issues. The paper further provides a detailed description of how MFA can be used for securing various smart city entities and services. A new concept of blockchain-based multi-factor authentication named "BAuth-ZKP" for securing smart city transactions is presented in the paper. The concept focuses on developing smart contracts between the participating entities within the smart city and performing the transactions with zero knowledge proof (ZKP)-based authentication in a secure and privacy-preserved manner. Finally, the future prospects, developments, and scope of using MFA in smart city ecosystem are discussed.

Erasure Codes for Cold Data in Distributed Storage Systems

Replication and erasure codes are always used for storing large amounts of data in distributed storage systems. Erasure code technology can maximize the storage space of distributed storage systems as well as guaranteeing their availability and reliability, but it will decrease the performance of the system when encoding and decoding. Since cold data do not require high real-time data availability, we focus on the cold data using erasure codes. We propose a new erasure code process named NewLib code based on the Liberation code, which designs the data alignment after stripping the encoding data. The NewLib code improves the performance of reading and writing in the distributed storage systems. At the same time, we developed a node scheduling scheme called N-Schedule, which divides the data nodes into multiple virtual nodes according to the storage space and computing power. The virtual nodes are dispersed into a hash ring by a consistent hash to construct a fully symmetric and decentralized hash ring in order to achieve uniform data distribution and task scheduling. The experimental results show our scheme can improve the performance of the system.

Urban Distribution and Evolution of the Yangtze River Economic Belt from the Perspectives of Urban Area and Night-Time Light

Research on urban development patterns and urban sprawl in the Yangtze River Economic Belt (YREB) has received wide attention. However, existing research has always made use of statistical data, which are not often available. Considering the high availability of satellite data, this study attempts to combine two satellite-acquired indexes, including urban area and night-time light, to evaluate the urban development of the YREB during 2012–2019. The methods included using growth index, rank-size law, and the Markov transition matrix, as well as constructing urban night-time light density and unbalanced index of night-time light, derived from the Gini Index. Some important patterns were revealed. Firstly, the three reaches (Upper Reaches, Middle Reaches, and Lower Reaches) in the YREB have all shown rapid growth in urban area and night-time light, and they all have increased in urban density. Secondly, from the perspective of regional disparity, the Upper Reaches have the highest growth rate of the urban area, while the Middle Reaches have the highest growth rate of night-time light; and the Upper Reaches have more urban sprawl, while the Middle Reaches have shown more compact growth. Thirdly, higher urban density is related to more balanced development across cities. Our study suggests new knowledge can be obtained by combining the two indexes for understanding urban development in the YREB.

Deep learning-based data privacy protection in software-defined industrial networking

The industrial Internet connects equipment to the network and utilizes the data generated to assist businesses. Industrial big data is the result of data accumulation; thus, the industrial Internet has to adopt new technologies—namely, software-defined industrial networks (SDIN) —to keep up with these developments. This study suggests a deep differential privacy data protection algorithm based on SDIN. The deep learning model is analyzed and integrated with differential privacy to provide the process framework for the deep differential privacy data protection algorithm. The equivalent model of the generative adversarial network is used to allow the attacker to obtain the optimal fake samples. The balance between dataset availability and privacy protection is achieved by implementing parameter tuning on the deep differential privacy model. The experimental results show that the proposed algorithm has strong industrial data privacy protection and high data availability and can effectively guarantee the privacy security of industrial data.

Intermediate data fault-tolerant method of cloud computing accounting service platform supporting cost-benefit analysis

This study mainly aims at the intermediate data fault-tolerant method of cloud computing accounting service platform supporting cost-benefit analysis, which aims at providing cost-benefit analysis function for the platform and strengthening the fault-tolerant ability of the intermediate data of the platform. The invention discloses a method for constructing an enterprise cloud computing accounting service platform. Collect the internal and external accounting service system data of an enterprise by using an accounting service platform network environment provided by a cloud service provider. Transmit the data to a data processing and storage layer data warehouse for storage after data cleaning, extraction and processing. Then, call the collected data through the data processing and storage layer to analyze the transaction cost-benefit of the enterprise. The intermediate data fault-tolerant model is constructed. After being solved by the ant colony algorithm, the intermediate data generated in the process of cost-benefit analysis and other accounting services are fault-tolerant processed. Finally, the platform accounting service results are output to the interactive interface through the data output display layer. The highest data availability probability of the method proposed in this study is 0. 98, which indicates that the method has high data availability after fault-tolerant processing, and can effectively realize the interaction with users. The experimental analysis shows that the method proposed in this study can effectively analyze the transaction costs and benefits of enterprises. The probability of data availability after fault-tolerant processing is higher, and a load of reading and writing is lower.

Handling security issues by using homomorphic encryption in multi-cloud environment

Taking advantage of the high performance and powerful data processing capabilities of cloud computing technology, externalizing data to the cloud platform is considered as an inevitable trend in the digital field today. However, ensuring the security and privacy of data remains a major challenge. To overcome this drawback, a multi-cloud platform is proposed to improve privacy and high availability of data. A multi-cloud platform that integrates public, private, and managed clouds with a single user interface. Cloud-hosted data is distributed among different data centers in a multi-cloud environment based on cloud reliability and data sensitivity. In terms of security, current encryption algorithms are considered to be very efficient, but it requires a lot of resources to handle this, which is expensive and time consuming. In addition, they also make the data impossible to process without first decoding. To be specific, traditional public key encryption requires data to be decrypted before it can be analyzed or manipulated. In contrast, homomorphic encryption is an encryption method that allows data to be encrypted while it is being processed and manipulated. It allows user or a third party, which can be cloud provider, to apply functions on encrypted data without revealing the data's values. In this paper, we explore existing multi-cloud-based security solutions using homomorphic encryption to identify open issues and opportunities for further enhancement.

Retracted] Storage Method for Medical and Health Big Data Based on Distributed Sensor Network

Monitoring and collecting medical data using embedded medical diagnostic devices with multiple sensors and sending these actual measured data to the corresponding health monitoring centers using multipurpose wireless networks to take necessary measures to coordinate with family medical service centers and regional medical service departments is a popular medical big data architecture. However, healthcare big data is characterized by large data volume, fast growth, multimodality, high value and privacy, etc. How to organize and manage it in a unified and efficient way is an important research direction at present. In response to the problems of low balance and poor security in the storage of data collected by distributed sensor networks in healthcare systems, we propose a distributed storage algorithm for big data in healthcare systems. The platform adopts Hadoop distributed file system and distributed file storage framework as the healthcare big data storage solution, and implements data integration, multidimensional data query and analysis mining components based on Spark‐SQL data query tool, Spark machine learning algorithm library and its mining and analysis pipeline development, respectively. The distributed storage model of big data and three data storage levels are constructed using cloud storage architecture, and the data storage intensity as well as levels are calculated by high data access in the upper level, data connection in the middle level, and data archiving in the lower level according to the set known data granularity, odds, and elasticity to realize big data storage. It is experimentally verified that the above algorithm has high distribution balance and low load balance in the storage process.

АНАЛИЗ ПРИМЕНЕНИЯ ПРОГРАММНОГО ОБЕСПЕЧЕНИЯ ДЛЯ РАСЧЕТА ГИПЕРУПРУГИХ МАТЕРИАЛОВ

The article discusses the use of software for the nonlinear analysis of hyperelastic material used in the soft capture mechanisms for robots working with agricultural products. Soft capture elements are made of rubber-like materials. Such materials have the ability to retain their elastic properties at high strain. The research reviewed various data resources, particularly, the reports on the development of soft grip elements for robots used to work with agricultural products. Internet resources were also studied to analyse the availability of data on the software for the nonlinear analysis of hyperelastic materials. The review has found that Abaqus, Ansys, and Comsol multiphysics software packages are used for the nonlinear analysis of hyperelastic elements. Abaqus is the most commonly used tool, although the programme training materials are not easily accessible. The Ansys software package ranks second with the highest data availability in the Internet. The review has found that the number of studies into Comsol multiphysics programme is the smallest though the availability of information on the tool in Internet sources ranks second.

RFID Landslide Monitoring: Long-Term Outdoor Signal Processing and Phase Unwrapping

Localization of passive Radio-Frequency Identification (RFID) tags has been used to monitor landslide surface displacement since 5 years. This method, applied on slow displacements lower than 1cm per day, allows a high spatio-temporal resolution at a relatively low cost. With the feedback of the previous years, this paper proposes to summarize the various challenges encountered with the long-term outdoor RFID localization method, and presents data-processing solutions that were implemented to overcome these challenges. We propose a complex-smoothing unwrapping algorithm, a multi-frequency merging operation, as well as multi-tag and multi-antenna phase combining method. The concept of an unwrapping reference guide is presented and applied with groups of tags showing coherent displacements, or with absolute reference measurements. These approaches allow a higher data availability up to 38% for one site over multiple years, and a better phase unwrapping. Earth surface displacement monitoring with RFID proves to be a robust and accurate solution, with four equipped sites across France and Switzerland.

Performance Enhancement of Distributed System Using HDFS Federation and Sharding

In today's world, 2.5 exabytes of data are generated and processed by the IT industry and different organizations. Processing and managing such a massive volume of Big data is challenging. Hadoop(HDFS) is widely used framework for processing Bigdata. The limitation of HDFS is it can scale only the datanodes and not namenode where the metadata is managed. There is no scaling of name nodes in the present Hadoop architecture, so managing the metadata of the exabyte scale becomes an essential and challenging leading to tightly coupled block storage, namespace scalability, and performance bottleneck issues. The proposed approach dynamic federated metadata management (DFMM) architecture manages the metadata in the name node. DFMM configures multiple name nodes using HDFS federation technique, and metadata is partitioned into numerous name nodes using sharding technique. Sharding manages the metadata using locality-preserving hashing and consistent hashing methods. Locality-preserving hashing stores metadata in the most appropriate name node using the file's parent directory, and metadata is dynamically distributed among Name nodes to maintain load balancing. DFMM also maintains replicas for the high availability and reliability of data. Results showcase the superiority of the proposed DFMM architecture combined with the sharding technique is highly scalable and manages metadata efficiently, and can store up to 1 billion files with a namespace size of 400 GB, enhancing the throughput by 21% and is highly scalable as compared to existing techniques. The comparative study analysis of existing HDFS, and proposed DFMM architecture combined with sharding enhances the performance and manages the metadata more efficiently.

Health Data Availability Protection: Delta-XOR-Relay Data Update in Erasure-Coded Cloud Storage Systems

To achieve the high availability of health data in erasure-coded cloud storage systems, the data update performance in erasure coding should be continuously optimized. However, the data update performance is often bottlenecked by the constrained cross-rack bandwidth. Various techniques have been proposed in the literature to improve network bandwidth efficiency, including delta transmission, relay, and batch update. These techniques were largely proposed individually previously, and in this work, we seek to use them jointly. To mitigate the cross-rack update traffic, we propose DXR-DU which builds on four valuable techniques: (i) delta transmission, (ii) XOR-based data update, (iii) relay, and (iv) batch update. Meanwhile, we offer two selective update approaches: 1) data-delta-based update, and 2) parity-delta-based update. The proposed DXR-DU is evaluated via trace-driven local testbed experiments. Comprehensive experiments show that DXR-DU can significantly improve data update throughput while mitigating the cross-rack update traffic.

Design and Enhancement of Microstrip Patch Antenna Utilizing Mushroom Like-EBG for 5G Communications

Besides the considerable rise in wireless network usage and network-connected apps, this causes congestion, leading to a drop in data transmission speed. As a result of their high bandwidth availability and high data transfer speed, network operators recommended switching to higher frequencies. In communications via distance, the antenna plays a significant role in the transmission of the electrical radiated signal or the receiving of the electromagnetic signal. Many antennas can be utilized, one of which is the Microstrip Patch (MP) antenna. These antennas are very prevalent for their ease of configuration and design. On the other hand, it suffers from a lack of gain. In this research article, an MP antenna and MP antenna array are introduced and designed based on computer simulation technology antenna modelling software for 5G based-28 GHz applications. In order to optimise and enhance the simulated MP antenna parameters, the mushroom-like Electromagnetic Band Gap (EBG) is embedded with the MP antenna and MP antenna array to eliminate the presence of the surface waves that affecting negatively on the antenna gain. According to the simulation results, the gain is enhanced after utilising the mushroom-like EBG to be 6.48 dBi and significantly after applying the array configuration to be 11.9 dBi.

Optimized Energy-Flexible Operation Strategies of Air Conditioning Systems in Production Environments

Rising energy prices and regulatory requirements are forcing both large manufacturing companies and small and medium-sized enterprises to minimize costs and emissions associated with energy supply. To operate the considered systems more efficiently overall, numerous energy efficiency measures have already been successfully implemented in recent years. This is particularly the case for the cross-industry air conditioning systems that are the focus of this paper. However, due to the growing share of renewable energies and the resulting fluctuating electricity prices and CO2 emissions factors, energy flexibility measures are becoming increasingly relevant. In this context, flexibility measures allow industrial system operators to benefit from these fluctuations, primarily using energy storages and optimized operation strategies. However, the implementation of energy flexibility measures requires high data availability, a good understanding of the system, and flexibility in production planning. To be able to effectively use the existing energy flexibility potentials, air conditioning systems are examined, that contain widely used cross-section technologies. In this paper, a method to enable optimized energy-flexible operation of industrial air conditioning systems is presented. A common feature of manufacturing sites is a similar topology of the thermal supply infrastructure across industries. The complex overall system is divided into several subsystems, e.g., cooling supply system, ventilation system and the production hall. Production processes typically release waste heat into the indoor air of the production hall, which then must be cooled. By accounting for thermal and electrical output and optimizing the operation strategy using mathematical optimization, both operating costs and CO2 emissions associated with energy provision can be significantly reduced. The presented use case reveals that operational cost savings of up to 18 % and a reduction of CO2 emission associated with energy supply of up to 16 % can be realized by optimizing the cooling supply system on its own.

Efficient Heuristic Replication Techniques for High Data Availability in Cloud

Most social networks allow connections amongst many people based on shared interests. Social networks have to offer shared data like videos, photos with minimum latency to the group, which could be challenging as the storage cost has to be minimized and hence entire data replication is not a solution. The replication of data across a network of read-intensive can potentially lead to increased savings in cost and energy and reduce the end-user’s response time. Though simple and adaptive replication strategies exist, the solution is non-deterministic; the replicas of the data need to be optimized to the data usability, performance, and stability of the application systems. To resolve the non-deterministic issue of replication, metaheuristics are applied. In this work, Harmony Search and Tabu Search algorithms are used optimizing the replication process. A novel Harmony-Tabu search is proposed for effective placement and replication of data. Experiments on large datasets show the effectiveness of the proposed technique. It is seen that the bandwidth saving for proposed harmony-Tabu replication performs better in the range of 3.57% to 18.18% for varying number of cloud datacenters when compared to simple replication, Tabu replication and Harmony replication algorithm.

Drone-Based Atmospheric Soundings Up to an Altitude of 10 km-Technical Approach towards Operations

Currently, the main in situ upper air database for numerical weather prediction relies on radiosonde and aircraft-based information. Typically, radiosondes are launched at specific sites daily, up to four times per day, and data are distributed worldwide via the GTS net. Aircraft observations are limited to frequent flight routes, and vertical profiles are provided in the vicinity of large cities. However, there are large areas with few radiosonde launches, in particular above the oceans and in the polar areas. In this article, the development and technical details of the unmanned aerial system LUCA (Lightweight Unmanned high Ceiling Aerial system) are described. LUCA has the potential to complement radiosonde and aircraft-based observations up to 10 km in altitude. The system ascends and descends (by electrical power) in spiral trajectories and returns to the launching site. This article discusses the requirements for obtaining high data availability under mid-European and Antarctic conditions, with highly automated take-offs and landings under high surface winds, the capacity to deal with icing, and the ability to operate under high wind speeds. The article presents technical solutions for the design and construction of the system and demonstrates its potential.

A Method of Transparent Graceful Failover in Low Latency Stateful Microservices

Microservice architecture is a preferred way to build applications. Being flexible and loosely coupled, it allows to deploy code at a high pace. State, or, in other words, data is not only a commodity but crucial to any business. The high availability and accessibility of data enables companies to remain competitive. However, maintaining low latency stateful microservices, for example, performing updates, is difficult compared to stateless microservices. Making changes to a stateful microservice requires a graceful failover, which has an impact on the availability budget. The method of graceful failover is proposed to improve availability of a low latency stateful microservice when performing maintenance. By observing database connection activity and forcefully terminating idle client connections, the method allows to redirect database requests from one node to another with negligible impact on the client. Thus, the proposed method allows to keep the precious availability budget untouched while performing maintenance operations on low latency stateful microservices. A set of experiments was performed to evaluate stateful microservice availability during failover and to validate the method. The results have shown that near-zero downtime was achieved during a graceful failover.

Impact of Cryptographic Key on Scalable Computing

The research has contributed to the development of the cryptographic control on the proposed RTS that aims to determine the high-performance computing at optimal cost and time to be invested into dynamic cryptographic control that decides on the major components of real-time operating system resources. Furthermore, the mechanism optimizes the cost, and resources are supposed to optimize the operating system risks. We have to optimize the technology and resource cost and maximizes the productivity and business (throughput) while improving the high performance of the operating system as per business requirement for the multiple locations. This proposed cryptographic control on the real-time system provides high computational services around the clock. The objective should be defined in such a way that the processor, memory, and encryption key are always utilized at minimal cost with high availability of data and services as per business and resource management.

Long-term analysis of the urban heat island effect using multisource Landsat images considering inter-class differences in land surface temperature products

It is imperative to quantitatively analyze the long-term temporal and spatial characteristics of the urban heat island (UHI) effect on cities for applications, such as urban expansion and environmental protection. Owing to the high spatial resolution and availability of long time-series data, remote sensing images from Landsat satellites are widely used for land surface temperature (LST) retrieval. However, limited by the satellite revisit cycle and image quality, the use of multisource Landsat images in a long-term study of the UHI effect is inevitable. Nonetheless, owing to the differences among multisource sensors, such as Landsat-7 and Landsat-8, there may be apparent deviations in the LST results retrieved from different sensor data, which are obtained from the same area and under similar circumstances. Consequently, it is necessary to build a relationship between the LST results generated from multisource Landsat sensors for future research on the UHI effect.In this study, Shenzhen city was studied to explore the fitting relationship between the corresponding LST products from Landsat-7 and Landsat-8 images obtained from adjacent dates with similar climatic conditions. Furthermore, factors affecting the fitting models, such as land cover types, seasonal and inter-annual differences, were analyzed. The constructed fitting model had a strong relationship with land cover types but a relatively weak relationship with seasonal and inter-annual differences; this indicates that a pseudo Landsat-8-based LST product can be generated from a Landsat-7-based LST product using a model fitted by a Landsat-7/8 pair obtained from adjacent years (or different seasons). Finally, by considering the consistency between LST products from multisource Landsat images, the spatiotemporal variations in the UHI effect in Shenzhen can be accurately explored using long time-series data.

Convolution-LSTM-Based Mechanical Hard Disk Failure Prediction by Sensoring S.M.A.R.T. Indicators

The traditional Infrastructure as a Service (IaaS) cloud platform tends to realize high data availability by introducing dedicated storage devices. However, this heterogeneous architecture has high maintenance cost and might reduce the performance of virtual machines. In homogeneous IaaS cloud platform, servers in the platform would uniformly provide computing resources and storage resources, which effectively solve the above problems, although corresponding mechanisms need to be introduced to improve data availability. Efficient storage resource availability management is one of the key methods to improve data availability. As mechanical hard disk is the main way to realize data storage in IaaS cloud platform at present, timely and accurate prediction of mechanical hard disk failure and active data backup and migration before mechanical hard disk failure would effectively improve the data availability of IaaS cloud platform. In this paper, we propose an improved algorithm for early warning of mechanical hard disk failures. We first use the Relief feature selection algorithm to perform parameter selection. Then, we use the zero-sum game idea of Generative Adversarial Networks (GAN) to generate fewer category samples to achieve a balance of sample data. Finally, an improved Long Short-Term Memory (LSTM) model called Convolution-LSTM (C-LSTM) is used to complete accurate detection of hard disk failures and achieve fault warning. We evaluate several models using precision, recall, and Area Under Curve (AUC) value, and extensive experiments show that our proposed algorithm outperforms other algorithms for mechanical hard disk warning.