Eventual Consistency Research Articles

On the Monitoring of Decentralized Specifications

We introduce two complementary approaches to monitor decentralized systems. The first approach relies on systems with a centralized specification, i.e., when the specification is written for the behavior of the entire system. To do so, our approach introduces a data structure that (i) keeps track of the execution of an automaton (ii) has predictable parameters and size, and (iii) guarantees strong eventual consistency. The second approach defines decentralized specifications wherein multiple specifications are provided for separate parts of the system. We study two properties of decentralized specifications pertaining to monitorability and compatibility between specification and architecture. We also present a general algorithm for monitoring decentralized specifications. We map three existing algorithms to our approaches and provide a framework for analyzing their behavior. Furthermore, we present THEMIS, a framework for designing such decentralized algorithms and simulating their behavior. We demonstrate the usage of THEMIS to compare multiple algorithms and validate the trends predicted by the analysis in two scenarios: a synthetic benchmark and the Chiron user interface.

Using weaker consistency models with monitoring and recovery for improving performance of key-value stores

Consistency properties provided by most key-value stores can be classified into sequential consistency and eventual consistency. The former is easier to program with but suffers from lower performance whereas the latter suffers from potential anomalies while providing higher performance. We focus on the problem of what a designer should do if he/she has an algorithm that works correctly with sequential consistency but is faced with an underlying key-value store that provides a weaker (e.g., eventual or causal) consistency. We propose a detect-rollback based approach: The designer identifies a correctness predicate, say P, and continues to run the protocol, as our system monitors P. If P is violated (because the underlying key-value store provides a weaker consistency), the system rolls back and resumes the computation at a state where P holds.We evaluate this approach with graph-based applications running on the Voldemort key-value store. Our experiments with deployment on Amazon AWS EC2 instances show that using eventual consistency with monitoring can provide a 50–80% increase in throughput when compared with sequential consistency. We also observe that the overhead of the monitoring itself was low (typically less than 4%) and the latency of detecting violations was small. In particular, in a scenario designed to intentionally cause a large number of violations, more than 99.9% of violations were detected in less than 50 ms in regional networks (all clients and servers in the same Amazon AWS region) and in less than 3 s in global networks.We find that for some applications, frequent rollback can cause the program using eventual consistency to effectively stall. We propose alternate mechanisms for dealing with re-occurring rollbacks. Overall, for applications considered in this paper, we find that even with rollback, eventual consistency provides better performance than using sequential consistency.

NoSQL A Solution for Handling Big Data

The data produced nowadays is the Big data which is unstructured, semi-structured or structured in nature. It is difficult for SQL to handle such large amount of data with varied forms so NoSQL was introduced which gives many advantages over SQL. It is a schema less database which allows horizontal scaling, scalability and distributed framework. SQL is based on ACID properties (Atomicity, Consistency, Isolation, Durability) whereas NoSQL is based on BASE (Basic Availability, Soft state, Eventual consistency). This paper introduces the concepts of NoSQL, advantages of NoSQL over SQL, different types of NoSQL databases with special reference to the document store database, MongoDB. It explains in detail about the MongoDB and then experimentally evaluates the performance of the queries executed in SQL and NoSQL (MongoDB). The experiment conducted shows that NoSQL queries are executed faster as compared to SQL queries and can also handle huge amount of unstructured data very effortlessly and easily.

A fault-tolerant programming model for distributed interactive applications

Ubiquitous connectivity of web, mobile, and IoT computing platforms has fostered a variety of distributed applications with decentralized state. These applications execute across multiple devices with varying reliability and connectivity. Unfortunately, there is no declarative fault-tolerant programming model for distributed interactive applications with an inherently decentralized system model. We present a novel approach to automating fault tolerance using high-level programming abstractions tailored to the needs of distributed interactive applications. Specifically, we propose a calculus that enables formal reasoning about applications' dataflow within and across individual devices. Our calculus reinterprets the functional reactive programming model to seamlessly integrate its automated state change propagation with automated crash recovery of device-local dataflow and disconnection-tolerant distribution with guaranteed automated eventual consistency semantics based on conflict-free replicated datatypes. As a result, programmers are relieved of handling intricate details of distributing change propagation and coping with distribution failures in the presence of interactivity. We also provides proofs of our claims, an implementation of our calculus, and an empirical evaluation using a common interactive application.

A Review of Polyglot Persistence in the Big Data World

The inevitability of the relationship between big data and distributed systems is indicated by the fact that data characteristics cannot be easily handled by a standalone centric approach. Among the different concepts of distributed systems, the CAP theorem (Consistency, Availability, and Partition Tolerant) points out the prominent use of the eventual consistency property in distributed systems. This has prompted the need for other, different types of databases beyond SQL (Structured Query Language) that have properties of scalability and availability. NoSQL (Not-Only SQL) databases, mostly with the BASE (Basically Available, Soft State, and Eventual consistency), are gaining ground in the big data era, while SQL databases are left trying to keep up with this paradigm shift. However, none of these databases are perfect, as there is no model that fits all requirements of data-intensive systems. Polyglot persistence, i.e., using different databases as appropriate for the different components within a single system, is becoming prevalent in data-intensive big data systems, as they are distributed and parallel by nature. This paper reflects the characteristics of these databases from a conceptual point of view and describes a potential solution for a distributed system—the adoption of polyglot persistence in data-intensive systems in the big data era.

Consistency Models of NoSQL Databases

Internet has become so widespread that most popular websites are accessed by hundreds of millions of people on a daily basis. Monolithic architectures, which were frequently used in the past, were mostly composed of traditional relational database management systems, but quickly have become incapable of sustaining high data traffic very common these days. Meanwhile, NoSQL databases have emerged to provide some missing properties in relational databases like the schema-less design, horizontal scaling, and eventual consistency. This paper analyzes and compares the consistency model implementation on five popular NoSQL databases: Redis, Cassandra, MongoDB, Neo4j, and OrientDB. All of which offer at least eventual consistency, and some have the option of supporting strong consistency. However, imposing strong consistency will result in less availability when subject to network partition events.

Security of NoSQL Database Against Intruders

Background: The evolution of distributed web-based applications and cloud computing has brought about the demand to store a large amount of big data in distributed databases. Such efficient systems offer excessive availability and scalability to users. The new type of database resolves many new challenges especially in large-scale and high concurrency applications which are not present in the relational database. NoSQL refers to non-relational databases that are different from the Relational Database Management System. Objective: NoSQL has many features over traditional databases such as high scalability, distributed computing, lower cost, schema flexibility, semi or un-semi structural data and no complex relationship. Method: NoSQL databases are “BASE” Systems. The BASE (Basically Available, Soft state, Eventual consistency), formulates the CAP theorem the properties of which are used by BASE System. The distributed computer system cannot guarantee all of the following three properties at the same time that is consistency, availability and partition tolerance. Results: As progressively sharp big data is saved in NoSQL databases, it is essential to preserve higher security measures to ensure safe and trusted communication across the network. In this patent, we describe the security of NoSQL database against intruders which is growing rapidly. Conclusion: This patent also defines probably the most prominent NoSQL databases and describes their security aspects and problems.

NetMob: A Mobile Application Development Framework with Enhanced Large Objects Access for Mobile Cloud Storage Service

Mobile enterprise applications are primarily devel-oped for existing backend enterprise systems and some usage scenarios require storing of large files on the mobile devices.These files range from large PDFs to media files in various formats (MPEG videos).These files needs to be used offline , sometimes updated and shared among users. Present work studied different Mobile Backend as a service (M)BaaS platforms to understand techniques use to handle large file and found that many are either missing the feature or does not handle performance issues for large files. In this paper, we are proposing, NetMob, a mobile synchronization platform that allows resource-limited mobile devices to access large objects from the cloud. This framework is mainly focused on large file handling and has support for both table and objects data models that can be tuned for three consistency semantics, resembling strong, causal and eventual consistency. Experimental results conducted using representative workloads showed that NetMob can handle large files access with the size ranging from 100MB upto 1GB and is able to reduce sync time with object chunking in our experiment settings.

Scalable distributed collaborative editing for 3D models using conflict-free data structure

Consistent maintenance in collaborative editing is one of the most important problems in distributed systems. So many papers have proposed various approaches to this issue, but most of them are for documents, and there are a few for 3D models. We propose a data structure, called ChainVoxel, and operations on it for scalable distributed collaborative editing systems for 3D models. It is based on the conflict-free replicated data type (CRDT). ChainVoxel is implemented by a chained hash table for collision resolution while editing by multiple users. It guarantees the eventual consistency for updating a shared 3D model without mutual exclusion. We analysed our prototype regarding the size of ChainVoxel and the number of operations. We also conducted simulations to compare our prototype with a consensus-based implementation. Our prototype shows the scalability regarding the number of messages and steps for executing operations.

Impact of Adaptive Consistency on Distributed SDN Applications: An Empirical Study

Scalability of the control plane in a software-defined network (SDN) is enabled by means of decentralization of the decision-making logic, i.e., by replication of controller functions to physically or virtually dislocated controller replicas. Replication of a centralized controller state also enables the protection against controller failures by means of primary and backup replicas responsible for managing the underlying SDN data plane devices. In this paper, we investigate the effect of the deployed consistency model on scalability and correctness metrics of the SDN control plane. In particular, we compare the strong and eventual consistency, and make a case for a novel adaptive consistency approach. The existing controller platforms rely on either strong or eventual consistency mechanisms in their state distribution. We show how an adaptive consistency model offers the scalability benefits in terms of the total request-handling throughput and response time, in contrast to the strong consistency model. We also outline how the adaptive consistency approach can provide for correctness semantics that are unachievable with the eventual consistency paradigm in practice. The adaptability of our approach provides a balanced and tunable tradeoff of scalability and correctness for the SDN application implemented on top of the adaptive framework. To validate our assumptions, we evaluate and compare the different approaches in an emulated testbed with an example of a load balancer controller application. The experimental setup comprises up to five extended OpenDaylight controller instances and two network topologies from the area of service provider and data center networks.

АРХІТЕКТУРА МЕХАНІЗМІВ ОБРОБКИ ДАНИХ ТА СИНХРОНІЗАЦІЯ МОДУЛІВ У ВИСОКОНАВАНТАЖЕНИХ СИСТЕМАХ SMART CITY


 
 
 
 The study is aimed to determine the main problem during developing and maintaining services of high load Smart City system and propose solutions for achieving eventual consistency between services. The main problem with the services and databases is a significant amount of requests which is produced by millions of devices and how to process and store it quite fast. Low latency, high scalability and failure resistance should be the main characteristic of the system. That is why choosing a database, a right strategy for database replicas, service synchronization and its monitoring are basic problems which must be solved first. There are several architecture and database types which are already used in more simple systems. Key aspect needs to be resolved – how to synchronize data between multiple services in Smart City system. To solve the problems we need to redevelop already existing technology which is used for more simple problems, join them and apply on new solution.
 
 
 

Reactive caching for composed services: polling at the speed of push

Sometimes, service clients repeat requests in a polling loop in order to refresh their view. However, such polling may be slow to pick up changes, or may increase the load unacceptably, in particular for composed services that disperse over many components. We present an alternative reactive polling API and reactive caching algorithm that combines the conceptual simplicity of polling with the efficiency of push-based change propagation. A reactive cache contains a summary of a distributed read-only operation and maintains a connection to its dependencies so changes can be propagated automatically. We first formalize the setting using an abstract calculus for composed services. Then we present a fault-tolerant distributed algorithm for reactive caching that guarantees eventual consistency. Finally, we implement and evaluate our solution by extending the Orleans actor framework, and perform experiments on two benchmarks in a distributed cloud deployment. The results show that our solution provides superior performance compared to polling, at a latency that comes close to hand-written change notifications.

On the Synchronization Bottleneck of OpenStack Swift-Like Cloud Storage Systems

As one type of the most popular cloud storage services, OpenStack Swift and its follow-up systems replicate each object across multiple storage nodes and leverage object sync protocols to achieve high reliability and eventual consistency . The performance of object sync protocols heavily relies on two key parameters: $r$ (number of replicas for each object) and $n$ (number of objects hosted by each storage node). In existing tutorials and demos, the configurations are usually $r=3$ and $n by default, and the sync process seems to perform well. However, we discover in data-intensive scenarios, e.g., when $r>3$ and $n\gg 1,000$ , the sync process is significantly delayed and produces massive network overhead, referred to as the sync bottleneck problem . By reviewing the source code of OpenStack Swift, we find that its object sync protocol utilizes a fairly simple and network-intensive approach to check the consistency among replicas of objects. Hence in a sync round, the number of exchanged hash values per node is $\Theta (n\times r)$ . To tackle the problem, we propose a lightweight and practical object sync protocol, LightSync , which not only remarkably reduces the sync overhead, but also preserves high reliability and eventual consistency. LightSync derives this capability from three novel building blocks: 1) Hashing of Hashes , which aggregates all the $h$ hash values of each data partition into a single but representative hash value with the Merkle tree; 2) Circular Hash Checking , which checks the consistency of different partition replicas by only sending the aggregated hash value to the clockwise neighbor; and 3) Failed Neighbor Handling , which properly detects and handles node failures with moderate overhead to effectively strengthen the robustness of LightSync. The design of LightSync offers provable guarantee on reducing the per-node network overhead from $\Theta (n\times r)$ to $\Theta (\frac{n}{h})$ . Furthermore, we have implemented LightSync as an open-source patch and adopted it to OpenStack Swift, thus reducing the sync delay by up to 879 $\times$ and the network overhead by up to 47.5 $\times$ .

A novel CRDT-based synchronization method for real-time collaborative CAD systems

CRDT (Conflict-free Replicated Data Type) has been proposed as an alternative synchronization mechanism for collaborative text editing systems in recent years. However, CRDT-based synchronization methods for collaborative systems with sophisticated objects, such as collaborative CAD (Co-CAD) systems, are rarely investigated in previous literatures. How well CRDT-based synchronization methods for Co-CAD systems could perform remains unknown. This paper presents a novel CRDT-based synchronization method to maintain eventual consistency for the feature-based CAD model. Firstly, three operation relations are defined as the dependency-conflict relation, the mutual exclusive relation and the compatible relation in context of the feature-based CAD systems. Secondly, a feature-based conflict detection mechanism is proposed to detect the three relations. Thirdly, a feature-based conflict resolution approach under the CRDT framework is presented to solve the conflicts. Fourthly, the time complexity and the space complexity are analyzed in theory. Finally, case studies of collaborative modeling procedures verify the correctness and feasibility of the proposed method.

Formalizing and Implementing Distributed Ledger Objects

Despite the hype about blockchains and distributed ledgers, formal abstractions of these objects are scarce1. To face this issue, in this paper we provide a proper formulation of a distributed ledger object. In brief, we de ne a ledger object as a sequence of records, and we provide the operations and the properties that such an object should support. Implemen- tation of a ledger object on top of multiple (possibly geographically dispersed) computing devices gives rise to the distributed ledger object. In contrast to the centralized object, dis- tribution allows operations to be applied concurrently on the ledger, introducing challenges on the consistency of the ledger in each participant. We provide the de nitions of three well known consistency guarantees in terms of the operations supported by the ledger object: (1) atomic consistency (linearizability), (2) sequential consistency, and (3) eventual consistency. We then provide implementations of distributed ledgers on asynchronous message passing crash- prone systems using an Atomic Broadcast service, and show that they provide eventual, sequen- tial or atomic consistency semantics respectively. We conclude with a variation of the ledger the validated ledger which requires that each record in the ledger satis es a particular validation rule.

New Requirements of Biodiversity Research for Metadata on Models and Sensors on the Internet of Things and Big Data Era

Important initiatives, such as the Convention on Biological Diversity's (CBD) Aichi targets, the United Nations' 2030 Agenda for Sustainable Development (and its Sustainable Development Goals) highlight the urgent need to stop the continuous and increasing loss of biodiversity. That requires an increase in the knowledge that will allow for sustainable use of natural resources. To accomplish that, detailed studies are needed to evaluate multiple species and regions. These studies demand great effort from professionals, searching for species and/or observing their behavior. In this case, the use of new monitoring devices could be beneficial in data collection and identification, optimizing the specialist effort to detect and observe species in-situ. With the advance of technology platforms for developing connected devices and sensors, associated with the evolution of the Internet of Things (IoT) concepts, and the advances of unmanned aerial vehicles (UAVs) and Wireless sensor networks (WSN), new scenarios in biodiversity studies are possible. The technology available now could allow studies applying relatively cheaper sensors with long-range (approx. 15 km), low power, low bit rate communication and up to 10-year battery life, using a Low Power Wide Area Network (LPWAN) and with capacity to run bio-acoustic or image processing detection. Platforms like Raspberry Pi or any other with signal processing capabilities can be applied (Hodgkinson and Young 2016). Sensor technologies protocols applied in IoT networks are usually simple and flexible. Common semantics and metadata definitions are necessary to extract information and representations to construct complex networks. Some of these metadata definitions can be adopted from the current Darwin Core schema. However, Darwin Core evolved based on enterprise technologies (i.e. XML) and relational database definitions, that usually need machines with significant bandwidth to transmit data. Today the technology scenario is taking another route, going from centralized to distributed architectures, occasionally applying non-relational and distributed databases, ready to deal with synchronization and eventual consistency problems. These distributed databases are usually employed to construct complex networks, where relation restrictions are not mandatory or, sometimes, even desired (Baggio et al. 2016). With these new techniques becoming a reality in biodiversity conservation studies, new metadata definitions are necessary. Those new metadata need to standardize and create a shared vocabulary that includes requirements for devices information exchange, data analytics, and model generation. Also, these new definitions could aggregate the Essential Biodiversity Variables (EBVs) concepts, that aim to identify the minimum of variables that can be used to inform scientists, managers and decision makers (Haase et al. 2018). For this reason, we propose the insertion of EBV definitions in the construction of sensor integration metadata and models characterization inside the Darwin Core metadata definitions (Fig. 1).

Scalable distributed control plane for On-line social networks support cognitive neural computing in software defined networks

Though most of the current proposed distributed control planes maintain strong consistency among their controllers, this paper argues the strong consistency is not a prerequisite and proposes an Event Coordination System (ECS) that enables an efficient event replaying system and a distributed control plane (DisCon) using this event replaying system to construct eventually consistent global network topologies among its controllers without sacrificing scalability. Our ECS implements a novel request handling procedure that ensures a firstly received write request is firstly multi-casted, notified, and updated, so thus our DisCon can maximally ensure the same time sequence in which topology events get updated at different controllers and the constructed topologies can reflect the real network change in practice. We highlight the major mechanisms used, discuss the major causes of this eventual consistency, estimate the inconsistency window among controllers, and show how this eventual consistency does not make a big difference in supporting network applications. Experiments are conducted to evaluate our ECS and DisCon. The results show our DisCon has a larger event replay throughput and a lower event converging delay than HyperFlow, and larger flow setup rate and lower flow setup delay than most of the current distributed control planes.

Eventual Consistency: Origin and Support

Eventual consistency is demanded nowadays in geo-replicated services that need to be highly scalable and available. According to the CAP constraints, when network partitions may arise, a distributed service should choose between being strongly consistent or being highly available. Since scalable services should be available, a relaxed consistency (while the network is partitioned) is the preferred choice. Eventual consistency is not a common data-centric consistency model, but only a state convergence condition to be added to a relaxed consistency model. There are still several aspects of eventual consistency that have not been analysed in depth in previous works: 1. which are the oldest replication proposals providing eventual consistency, 2. which replica consistency models provide the best basis for building eventually consistent services, 3. which mechanisms should be considered for implementing an eventually consistent service, and 4. which are the best combinations of those mechanisms for achieving different concrete goals. This paper provides some notes on these important topics.

Research on highly available storage system for open virtual experiment platform

Open teaching platform for virtual experiment can overcome some disadvantages of traditional virtual experiment systems such as heterogeneity of experiments and realise great interaction and availability. In this paper, a highly available data storage model for open virtual experiment platform is put forward, in which consistent hashing distribution, eventual consistency is used to realise this system and reading and writing control mechanism and exception handling method are adopted to ensure availability of system services. In this system, standardisation of experimental procedures and data stream are realised, based on which the efficiency of data communication is increased and development costs of virtual experiments are reduced.

Research on highly available storage system for open virtual experiment platform

Open teaching platform for virtual experiment can overcome some disadvantages of traditional virtual experiment systems such as heterogeneity of experiments and realise great interaction and availability. In this paper, a highly available data storage model for open virtual experiment platform is put forward, in which consistent hashing distribution, eventual consistency is used to realise this system and reading and writing control mechanism and exception handling method are adopted to ensure availability of system services. In this system, standardisation of experimental procedures and data stream are realised, based on which the efficiency of data communication is increased and development costs of virtual experiments are reduced.