High Data Availability Research Articles

Accelerated Map Matching for GPS Trajectories

The processing and analysis of large-scale journey trajectory data is becoming increasingly important as vehicles become ever more prevalent and interconnected. Mapping these trajectories onto a road network is a complex task, largely due to the inevitable measurement error generated by GPS sensors. Past approaches have had varying degrees of success, but achieving high accuracy has come at the expense of performance, memory usage, or both.In this paper, we solve these issues by proposing a map matching algorithm based on Hidden Markov Models (HMM). The proposed method is shown to be more efficient when compared against a traditional HMM based map matching method, whilst maintaining high accuracy and eschewing any requirements for CPU-intensive and memory-expensive pre-processing. The proposed algorithm offers a method for significantly accelerating transition-probability calculations using instances of high data-availability, which have previously been a large bottleneck in map matching algorithm performance. It is shown that this can be accomplished with the application of road-network segmentation combined with a spatially-aware heuristic. Experiments are performed using two different datasets, with over 9 hours of GPS samples. We show that the proposed framework is able to offer a reduction in run-time of over 90% with no significant effect on the algorithm’s accuracy when compared against the traditional HMM approach.

Differential Privacy Location Protection Method Based on the Markov Model

Location‐based services (LBS) have become an important research area with the rapid development of mobile Internet technology, GPS positioning technology, and the widespread application of smart phones and social networks. LBS can provide convenience and flexibility for the users’ daily life, but at the same time, it also brings security risks to the users’ privacy. Untrusted or malicious LBS servers can collect users’ location data through various ways and disclose it to the third party, thus causing users’ privacy leakage. In this paper, a differential privacy location protection method based on the Markov model for user’s location privacy is proposed. Firstly, the transition probability matrix between states of the n‐order Markov model is used to predict the occurrence state and development trend of events; thereby, the user’s location is predicted, and then a location prediction algorithm based on the Markov model (LPAM) is proposed. Secondly, a location protection algorithm based on differential privacy (LPADP) is proposed, in which location privacy tree (LPT) is constructed according to the location data and the difficulty of retrieval, the two nodes with the largest predicted value of LPT are allocated with a reasonable privacy budget, and Laplace noise is added to protect location privacy. Theoretical analysis and experimental results show that the proposed method not only meets the requirements of differential privacy and protects location privacy effectively but also has high data availability and low time complexity.

CAREDAS: a Comprehensive Architecture for a Redundant and Evolutive Data Acquisition System for JHR reactor

A new material testing reactor Jules Horowitz Reactor is under construction at CEA Cadarache. The materials to be irradiated will be placed into experimental devices around the reactor. Process and measurements of experimental devices will be carried out by command control. A data acquisition system having processing performances will be associated to the programmable logic controller. The challenge is to design and realize for twenty experiment devices a high availability data acquisition system architecture for 50 years of sustainability. The real time target will achieve 24/7 data acquisition and real time processing. This scalable architecture could be use as well for JHR experimental devices with high availability as for testbed. This architecture could be run on a standalone station for a measuring bench or deployed on cluster for high availability. CAREDAS’s design is modular and use proven widely used open source solutions. All parts are independent from each other and can be replaced with another technology with the same functionalities. This ensures sustainability and control of software sources.

Differential Co-Expression Analyses Allow the Identification of Critical Signalling Pathways Altered during Tumour Transformation and Progression.

Biological systems respond to perturbations through the rewiring of molecular interactions, organised in gene regulatory networks (GRNs). Among these, the increasingly high availability of transcriptomic data makes gene co-expression networks the most exploited ones. Differential co-expression networks are useful tools to identify changes in response to an external perturbation, such as mutations predisposing to cancer development, and leading to changes in the activity of gene expression regulators or signalling. They can help explain the robustness of cancer cells to perturbations and identify promising candidates for targeted therapy, moreover providing higher specificity with respect to standard co-expression methods. Here, we comprehensively review the literature about the methods developed to assess differential co-expression and their applications to cancer biology. Via the comparison of normal and diseased conditions and of different tumour stages, studies based on these methods led to the definition of pathways involved in gene network reorganisation upon oncogenes’ mutations and tumour progression, often converging on immune system signalling. A relevant implementation still lagging behind is the integration of different data types, which would greatly improve network interpretability. Most importantly, performance and predictivity evaluation of the large variety of mathematical models proposed would urgently require experimental validations and systematic comparisons. We believe that future work on differential gene co-expression networks, complemented with additional omics data and experimentally tested, will considerably improve our insights into the biology of tumours.

A Low-Cost Multi-Failure Resilient Replication Scheme for High-Data Availability in Cloud Storage

Data availability is one of the most important performance factors in cloud storage systems. To enhance data availability, replication is a common approach to handle the machine failures. However, previously proposed replication schemes cannot effectively handle both correlated and non-correlated machine failures, especially while increasing the data availability with limited resources. The schemes for correlated machine failures must create a constant number of replicas for each data object, which often neglects diverse data popularities and does not utilize the resource to maximize the expected data availability. Also, the previous schemes neglect the consistency maintenance cost and the storage cost caused by replication. It is critical for cloud providers to maximize data availability (hence minimize SLA violations) while minimizing costs caused by replication in order to maximize the revenue. In this paper, we build a nonlinear integer programming model to maximize data availability in both types of failures, and therefore minimize the cost caused by replication. Based on the model's solution for the replication degree of each data object, we propose a low-cost multi-failure (correlated and non-correlated machine failures) resilient replication scheme (MRR). MRR can effectively handle both correlated and non-correlated machine failures, considers data popularities to enhance data availability, and also tries to minimize consistency maintenance and storage cost. Extensive numerical results from trace parameters and experiments from real-world Amazon S3 demonstrate that MRR achieves high data availability, low data loss probability and low consistency maintenance and storage costs when compared to previous replication schemes.

Modified Data Storage and Replication Mechanism with Frequent Use-Case Based Indexing

The real time applications demands high speed and reliable data access from the remote database. An effective logical data management strategy that handles simultaneous connections with better performance negotiation is inevitable. This work considers an e-health care application that proposes MongoDB based modified indexing and performance tuning methods. To cope with certain high frequency use case and its performance mandates, a flexible and efficient logical data management may be preferred. By analysing the data dependency, data decomposition concerns and the performance requirements of the specific use case of the medical application, a logical schema may be customized on an ala-carte basis. This work focused on the flexible logical data modeling schemes and its performance factors of the NoSql DB. The efficiency of unstructured data base management in storing and retrieving the e-health care data was analysed with a web based tool. To enable faster data retrieval and query processing over the distributed nodes, a Spark based storage engine was built on top of the MongoDB based data storage management. With Spark tool, the database has been made distributed as master–slave structures with suitable data replication mechanisms. In such distributed database the fail-over also implemented with the suitable replication mechanism. This work considered MongoDB based flexible schema modeling and Spark based distributed computation with multiple chunks of data. The flexible data modeling scheme with MongoDB with the on-demand Spark based computation framework was proposed. To facilitate the eventual consistency, scalability aspects of the e-health care applications, use case based indexing was proposed. With the effective data management, faster query processing the horizontal scalability has been increased. The overall efficiency and scalability of the proposed logical data management approach was analysed. Through the simulation studies, the proposed approach has been claimed to boost the performance of the bigdata based application to a considerable extent.

Future DNA computing device and accompanied tool stack: Towards high-throughput computation

DNA Computing is still at its infant stage since its emergence. Multiple aspects of DNA computing have been studied but most of the research results have not been applied to the reality. It has been proved to exhibit high data storage density and support efficient random data access. It also shows the potential to provide alternative facilities for general computing. Especially, its natural double-helix structure enables it to be the best fit for high-throughput parallel computing. However, the underlying rationale of DNA computing is different from the existing electronic computing devices. Therefore, the popularity of DNA computing device is limited by its accessibility. We propose our designs for high-throughput DNA computing including DNA computing circuits, system architecture, and its compiler. We also demonstrate its feasibility using a simple example through simulation experiments. The objective of this framework is to bridge the gap between the existing computer developer community and the DNA computing biotechnology.

Are flood damage models converging to “reality”? Lessons learnt from a blind test

Abstract. Effective flood risk management requires a realistic estimation of flood losses. However, available flood damage estimates are still characterized by significant levels of uncertainty, questioning the capacity of flood damage models to depict real damages. With a joint effort of eight international research groups, the objective of this study was to compare, in a blind-validation test, the performances of different models for the assessment of the direct flood damage to the residential sector at the building level (i.e. microscale). The test consisted of a common flood case study characterized by high availability of hazard and building data but with undisclosed information on observed losses in the implementation stage of the models. The nine selected models were chosen in order to guarantee a good mastery of the models by the research teams, variety of the modelling approaches, and heterogeneity of the original calibration context in relation to both hazard and vulnerability features. By avoiding possible biases in model implementation, this blind comparison provided more objective insights on the transferability of the models and on the reliability of their estimations, especially regarding the potentials of local and multivariable models. From another perspective, the exercise allowed us to increase awareness of strengths and limits of flood damage modelling, which are summarized in the paper in the form of take-home messages from a modeller's perspective.

Database Performance Management in Cloud

Managing large chain of Hotels and ERP database comprises of core areas such as HRMS & PIP.HRMS (Human Resource Management System), which further includes areas such as Soft Joining, Promotion, Transfer, Confirmation, Leave Attendance and Exit, etc. PIP (Payroll Information Portal), wherein employees can view their individual Salary details, submit investment declaration, Reimbursement claim & CTC structuring, etc. Management of Large Chain of Hotels and ERP Database in AWS Cloud involves continuous monitoring with regards to the areas such as Performance of resource usages and optimization techniques relating to the use of PL/SQL. High Availability (HA) of data is accomplished through the Backup and Recovery mechanism and security of the data by Encryption & Decryption mechanism.

Zigzag-Decodable Reconstruction Codes With Asymptotically Optimal Repair for All Nodes

Zigzag-decodable codes have been proposed for distributed storage systems to achieve fast decoding of uncoded data packets through the iterative decoding of data bits from coded packets. To maintain high data availability, it is critical to minimize the repair bandwidth by downloading the least amount of bits for repairing any lost packet. In this work, we propose zigzag-decodable reconstruction (ZDR) codes which achieve asymptotically minimum repair bandwidth for repairing a single node, while preserving the high computational efficiency due to zigzag decoding. We present two explicit constructions of ZDR codes such that any node of ZDR codes can be repaired with asymptotically minimum repair bandwidth. The first construction is based on the well-designed encoding matrix and a generic transformation, while the second construction is designed by recursively employing the proposed generic transformation for any existing zigzag-decodable code. Moreover, we show that the proposed two classes of ZDR codes can be decoded by the zigzag decoding algorithm and have less computational complexity than the existing codes with asymptotically or exactly minimum repair bandwidth.

Continuous Forest Monitoring Using Cumulative Sums of Sentinel-1 Timeseries

Forest degradation is recognized as a major environmental threat on a global scale. The recent rise in natural and anthropogenic destruction of forested ecosystems highlights the need for developing new, rapid, and accurate remote sensing monitoring systems, which capture forested land transformations. In spite of the great technological advances made in airborne and spaceborne sensors over the past decades, current Earth observation (EO) change detection methods still need to overcome numerous limitations. Optical sensors have been commonly used for detecting land use and land cover changes (LULCC), however, the requirement of certain technical and environmental conditions (e.g., sunlight, not cloud-coverage) restrict their use. More recently, synthetic aperture radar (SAR)-based change detection approaches have been used to overcome these technical limitations, but they commonly rely on static detection approaches (e.g., pre and post disturbance scenario comparison) that are slow to monitor change. In this context, this paper presents a novel approach for mapping forest structural changes in a continuous and near-real-time manner using dense Sentinel-1 image time-series. Our cumulative sum–spatial mean corrected (CUSU-SMC) algorithm approach is based on cumulative sum statistical analysis, which allows the continuous monitoring of radar signal variations, derived from forest structural change. Taking advantage of the high data availability offered by the Sentinel-1 (S-1) C-band constellation, we used an S-1 ground range detected (GRD) dual (VV, VH) polarization timeseries, formed by a total of 84 images, to monitor clear-cutting operations carried out in a Scottish forest during 2019. The analysis showed a user’s accuracy of 82% for the (conservative) detection approach. The use of a post-processing neighbor filter increased the detection performance to a user’s accuracy of 86% with an overall accuracy of 77% for areas of a minimum extent of 0.4 ha. To further validate the detection performance of the method, the CUSU-SMC change detector was tested against commonly-used pairwise change detection approaches for the same period. These results emphasize the capabilities of dense SAR time-series for environmental monitoring and provide a useful tool for optimizing national forest inventories.

Synergetic utilization of sentinel-1 SAR and sentinel-2 optical remote sensing data for surface soil moisture estimation for Rupnagar, Punjab, India

Due to improper agricultural and soil management, there has been a drop in crop yield over the last few years and food security has become a major issue. For a country like India, with a huge population to cater, the problem becomes more serious. Since the inception of remote sensing in scientific agriculture management, optical remote sensing along with field data has been used for soil health monitoring and mapping. SAR or microwave remote sensing has an all-weather and high temporal data availability which has found applications for various domains. For soil health studies of multiple soil classes and sub-classes of same type, both aerial and spaceborne SAR remote sensing is currently in use for a multitude of monitoring and parameter modelling approaches. This study utilizes Sentinel 1 A, C-band SAR remote sensing data with VV and VH polarization channels for surface soil moisture estimation for alluvial soil and its sub-types in Rupnagar of Punjab state in India. While Index based OLS Regression method for soil moisture estimation was done using backscatter from Sentinel-1A SAR data, it was validated using Normalized Differential Moisture Index (NDMI) generated from Sentinel 2 optical datasets. This approach though did not consider the actual soil moisture data from field yet gave a low Root Mean Squared Error (RMSE) of 0.5 and R2-statistics of 0.72 (72%) in training and testing phases. The Index based OLS (Ordinary Least Squares) Regression method for soil moisture estimation aims to establish a technique for cases when field data is either not available or the study area is not easily accessible. In the statistical approach with field data, the same OLS model, when replaced by on-field surface soil moisture data gave a RMSE of 1.9 and R2-statistics of 0.968 and 0.948 in training and testing phases respectively at 97.5% confidence level. The study is significant for using freely available optical and SAR remote sensing data parameters synergistically, in a simplified manner for surface soil moisture estimation. The results have comparable accuracies given by studies using commercial data and complex modelling approaches.

Dynamic Proof of Data Possession and Replication With Tree Sharing and Batch Verification in the Cloud

Cloud storage attracts a lot of clients to join the paradise. For a high data availability, some clients require their files to be replicated and stored on multiple servers. Because clients are generally charged based on the redundancy level required by them, it is critical for clients to obtain convincing evidence that all replicas are stored correctly and are updated to the up-to-date version. In this article, we propose a dynamic proof of data possession and replication (DPDPR) scheme, which is proved to be secure in the defined security model. Our scheme shares a single authenticated tree across multiple replicas, which reduces the tree's storage cost significantly. Our scheme allows for batch verification for multiple challenged leaves and can verify multiple replicas in a single batch way, which considerably save bandwidth and computation resources during audit process. We also evaluate the DPDPR's performance and compare it with the most related scheme. The evaluation results show that our scheme saves almost 66 percent tree's storage cost for three replicas, and obtains almost 60 and 80 percent efficiency improvements in terms of the overall bandwidth and computation costs, respectively, when three replicas are checked and each challenged with 460 blocks.

A privacy-preserving mechanism based on local differential privacy in edge computing

With the development of Internet of Things (IoT), the delay caused by network transmission has led to low data processing efficiency. At the same time, the limited computing power and available energy consumption of IoT terminal devices are also the important bottlenecks that would restrict the application of blockchain, but edge computing could solve this problem. The emergence of edge computing can effectively reduce the delay of data transmission and improve data processing capacity. However, user data in edge computing is usually stored and processed in some honest-but-curious authorized entities, which leads to the leakage of users' privacy information. In order to solve these problems, this paper proposes a location data collection method that satisfies the local differential privacy to protect users' privacy. In this paper, a Voronoi diagram constructed by the Delaunay method is used to divide the road network space and determine the Voronoi grid region where the edge nodes are located. A random disturbance mechanism that satisfies the local differential privacy is utilized to disturb the original location data in each Voronoi grid. In addition, the effectiveness of the proposed privacy-preserving mechanism is verified through comparison experiments. Compared with the existing privacy-preserving methods, the proposed privacy-preserving mechanism can not only better meet users' privacy needs, but also have higher data availability.

GGraph: An Efficient Structure-Aware Approach for Iterative Graph Processing

Many iterative graph processing systems have recently been developed to analyze graphs. Although they are effective from different aspects, there is an important issue that has not been addressed yet. A real-world graph follows the power-law property, in which a small number of vertices have high degrees (i.e., are connected to most other vertices in the graph). These vertices are called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hot-vertices</i> and usually require more iterations to converge. In the existing solutions, these hot-vertices may be allocated to many or even all graph partitions along with other vertices that are easy to converge. As the result, the partitions with hot-vertices have to be loaded repeatedly (and consequently the system suffers from high data access cost), although perhaps only a few vertices in these partitions are active. To cope with this issue, we develop an efficient open source graph partition manager, called GGraph, which can be integrated into the existing graph processing systems to efficiently support iterative graph processing, by taking into account the power-law property of the graph structure. It uses a novel graph repartitioning scheme with low overhead to dynamically partition the hot-vertices together, so as to avoid loading the inactive vertices in the same partition as the repeatedly processed hot-vertices. By such means, it not only enables less data access cost, but also enables the privileged processing of the hot-vertices. In order to further increase the convergence speed, a scheduling algorithm is further proposed in this work to prioritize the processing of the hot-vertices with low overhead. To demonstrate the efficiency of GGraph, we plug it into four state-of-the-art graph processing systems, i.e., Gemini, GraphChi, Chaos, and GridGraph, and experimental results show that GGraph improves their performance by up to 3.2 times, 3.8 times, 3.9 times, 3.5 times, respectively.

A popularity-aware reconstruction technique in erasure-coded storage systems

In this study, we develop a novel data reconstruction technique for parallel storage systems housed in modern data centers. We advocate for erasure-coded data storage systems to archive warm data (a.k.a., unpopular data), which attract a limited number of accesses or updates. Different from hot or cold data, warm data have to be treated in a distinctive way to optimize system performance and storage-space utilization. We pay particular attention to efficient data reconstruction in which faulty data nodes are rebuilt while responding to I/O requests. To achieve this goal, we employ two machine-learning algorithms to offer online data reconstruction in erasure coded storage systems. Our data reconstruction technique is conducive to recovering faulty nodes while boosting read performance for requests accessing data residing on the faulty nodes. Our system is reliant on a clustering mechanism to group files into multiple clusters, in each of which files share similar features. Furthermore, we implement a prediction module where a list of future popular data is projected by keeping track of historical I/O accesses. This popular-data list, in turn, provides predictions on files that are likely to be accessed in the not-too-distant future. The prediction module is responsible for computing similarities among users, thereby setting up priority levels of data blocks to be reconstructed. We implement our data reconstruction scheme in an erasure-coded parallel storage system to recover files with a guidance from the popular-data list. Our experimental results confirm that our system speeds up the data recovery of parallel storage systems while maintaining a high data access performance for on-line users.

Supercomputing and Secure Cloud Infrastructures in Biology and Medicine

The increasing amounts of healthcare data stored in health registries, in combination with genomic and other types of data, have the potential to enable better decision making and pave the path for personalized medicine. However, reaping the full benefits of big, sensitive data for the benefit of patients requires greater access to data across organizations and institutions in various regions. This overview first introduces cloud computing and takes stock of the challenges to enhancing data availability in the healthcare system. Four models for ensuring higher data accessibility are then discussed. Finally, several cases are discussed that explore how enhanced access to data would benefit the end user.

Scalable and Adaptive Data Replica Placement for Geo-Distributed Cloud Storages

In geo-distributed cloud storage systems, data replication has been widely used to serve the ever more users around the world for high data reliability and availability. How to optimize the data replica placement has become one of the fundamental problems to reduce the inter-node traffic and the system overhead of accessing associated data items. In the big data era, traditional solutions may face the challenges of long running time and large overheads to handle the increasing scale of data items with time-varying user requests. Therefore, novel offline community discovery and online community adjustment schemes are proposed to solve the replica placement problem in a scalable and adaptive way. The offline scheme can find a replica placement solution based on the average read/write rates for a certain period of time. The scalability can be achieved as 1) the computation complexity is linear to the amount of data items and 2) the data-node communities can evolve in parallel for a distributed replica placement. Furthermore, the online scheme is adaptive to handle the bursty data requests, without the need to completely override the existing replica placement. Driven by real-world data traces, extensive performance evaluations demonstrate the effectiveness of our design to handle large-scale datasets.

An Enhanced Security Framework for Secured Data Storage and Communications in Cloud Using ECC, Access Control and LDSA

In this paper, we propose an enhanced security framework for securing the cloud users data in the cloud environment. This new security framework consists of access control methodology, encryption/decryption technique and digital signature algorithms. Here, a new Elliptic Curve Cryptography based key generation algorithm is proposed for generating highly secured keys. Moreover, a new Identity based Elliptic curve Access Control mechanism (Id-EAC) is also proposed in this paper for restricting the cloud user’s data accessibility over the different kind of data. In addition, a new binary value based two phase encryption and decryption algorithm that referred the elliptic curve cryptographic based key values is introduced to secure the cloud user’s data in the cloud environment. Finally, a new modulo function based Lightweight Digital Signature Algorithm is also proposed to ensure the data integrity in this proposed security framework. This security framework is provided the high data security, accessibility and integrity for the user data. The experimental results show that the efficiency of the proposed algorithms in this security framework is proved as better than other existing algorithms in terms of security and performance.

Integrated Power Anomaly Defense: Towards Oversubscription-Safe Data Centers

Energy storage devices (e.g., batteries) are critical components for high-availability data center infrastructure today. Without resilient energy management of these devices, existing power-hungry data centers are largely unguarded targets for cyber criminals. Particularly for some of today's scale-out data centers, power infrastructure oversubscription unavoidably taxes the data center's backup energy resources (i.e., UPS), leaving very little room for dealing with power emergency. As a result, an attacker could manipulate the computing system to generate peak power demand and disrupt power-constrained server racks. This article aims at protecting data centers from malicious loads that seek to drain precious energy backup, overload server racks and compromise workload performance. We term such load as Elusive Power Peak (EPP) and demonstrate its basic three-phase attacking model. To defend against EPP, we propose IPAD, a remediation solution build on integrated software and hardware mechanisms. IPAD not only increases the attacking cost considerably by hiding vulnerable server racks from visible power peaks, but also strengthens the last line of defense against hidden power spikes with fine-grained power control strategy. We show that IPAD can effectively raise the bar of power-related attack, with reasonable design overhead.