Principles of Eventual Consistency

Distributed key-value (KV) stores are a rising alternative to traditional relational databases since they provide a flexible yet simple data model. Recent KV stores use eventual consistency to ensure fast reads and writes as well as high availability. Support for eventual consistency is however still very limited as typically only a handful of replicated data types are provided. Moreover, modern applications maintain various types of data, some of which require strong consistency whereas other require high availability. Implementing such applications remains cumbersome due to the lack of support for data consistency in today's KV stores. In this paper we propose Squirrel, an open implementation of an in-memory distributed KV store. The core idea is to reify distribution through consistency models and protocols. We implement two families of consistency models (strong consistency and strong eventual consistency) and several consistency protocols, including two-phase commit and CRDTs.

Consistency is one of the fundamental issues of distributed computing. There are many competing consistency models, with subtly different power in principle. In practice, the well known the Consistency-Availability-Partition Tolerance trade-off translates to difficult choices between fault tolerance, performance, and programmability. The issues and trade-offs are particularly vexing at scale, with a large number of processes or a large shared database, and in the presence of high latency and failure-prone networks. It is clear that there is no one universally best solution. Possible approaches cover the whole spectrum between strong and eventual consistency. Strong consistency (linearizability or serializability, achieved via total ordering) provides familiar and intuitive semantics but requires slow and fragile synchronization and coordination overheads. The unlimited parallelism allowed by weaker models such as eventual consistency promises high performance, but divergence and conflicts make it difficult to ensure useful application invariants, and metadata is hard to keep in check. The research and development communities are actively exploring intermediate models (replicated data types, monotonic programming, CRDTs, LVars, causal consistency, red-blue consistency, invariant- and proof-based systems, etc.), designed to improve efficiency, programmability, and overall operation without negatively impacting scalability.

Consistency is one of the fundamental issues of distributed computing. There are many competing consistency models, with subtly different power in principle. In practice, the well-known Consistency-Availability-Partition Tolerance trade-off translates to difficult choices between fault tolerance, performance, and programmability. The issues and trade-offs are particularly vexing at scale, with a large number of processes or a large shared database, and in the presence of high latency and failure-prone networks. It is clear that there is no one universally best solution. Possible approaches cover the whole spectrum between strong and eventual consistency. Strong consistency (total ordering via, for example, linearizability or serializability) provides familiar and intuitive semantics but requires slower and, in some contexts, fragile coordination. The unlimited parallelism allowed by weaker models such as eventual consistency promises high performance, but divergence and conflicts make it difficult to ensure useful application invariants, and meta-data is hard to keep in check. The research and development communities are actively exploring intermediate models (replicated data types, monotonic programming, CRDTs, LVars, causal consistency, red-blue consistency, invariant- and proof-based systems, etc.), designed to improve efficiency, programmability, and overall operation without negatively impacting scalability. This workshop aims to investigate the principles and practice of weak consistency models for large-scale, distributed shared data systems. It brings together theoreticians and practitioners from different horizons: system development, distributed algorithms, concurrency, fault tolerance, databases, language and verification, including both academia and industry.

Report on the first workshop on negative and null results in eScience

Strong consistency is widely used in systems such as relational databases. In a distributed system, strong consistency ensures that all clients observe consistent data updates atomically on all servers. However, such systems need to sacrifice availability when synchronization occurs.We propose a new consistency protocol, the observable atomic consistency protocol (OACP) to make write-dominant applications as fast as possible and as consistent as needed. OACP combines the advantages of (1) mergeable data types, specifically, convergent replicated data types, to reduce synchronization and (2) reliable total order broadcast to provide on-demand strong consistency. We also provide a high-level programming interface to improve the efficiency and correctness of distributed programming.We present a formal, mechanized model of OACP in rewriting logic and verify key correctness properties using the model checking tool Maude. Furthermore, we provide a prototype implementation of OACP based on Akka, a widely-used actor-based middleware. Our experimental evaluation shows that OACP can reduce coordination overhead compared to the state-of-the-art Raft consensus protocol. Our results also suggest that OACP increases availability through mergeable data types and provides acceptable latency for achieving strong consistency, enabling a principled relaxation of strong consistency to improve performance.

Consistency is one of the fundamental issues of distributed computing. There are many competing consistency models, with subtly different power in principle. In practice, the well known the Consistency-Availability-Partition Tolerance trade-off translates to difficult choices between fault tolerance, performance, and programmability. The issues and trade-offs are particularly vexing at scale, with a large number of processes or a large shared database, and in the presence of high latency and failure-prone networks. It is clear that there is no one universally best solution. Possible approaches cover the whole spectrum between strong and eventual consistency. Strong consistency (linearizability or serializability, achieved via total ordering) provides familiar and intuitive semantics but requires slow and fragile synchronization and coordination overheads. The unlimited parallelism allowed by weaker models such as eventual consistency promises high performance, but divergence and conflicts make it difficult to ensure useful application invariants, and metadata is hard to keep in check. The research and development communities are actively exploring intermediate models (replicated data types, monotonic programming, CRDTs, LVars, causal consistency, red-blue consistency, invariant- and proof-based systems, etc.), designed to improve efficiency, programmability, and overall operation without negatively impacting scalability. This workshop aims to investigate the principles and practice of consistency models for large-scale, distributed shared data systems. It will bring together theoreticians and practitioners from different horizons: system development, distributed algorithms, concurrency, fault tolerance, databases, language and verification, including both academia and industry.

The development of distributed systems requires developers to balance the need for consistency, availability, and partition tolerance. Conflict-free replicated data types (CRDTs) are widely used in eventually consistent systems to reduce concurrency control. However, CRDTs lack consistent totally-ordered operations which can make them difficult to use. In this paper, we propose a new consistency protocol, the observable atomic consistency protocol (OACP). OACP enables a principled relaxation of strong consistency to improve performance in specific scenarios. OACP combines the advantages of mergeable data types, specifically, convergent replicated data types, and reliable total order broadcast to provide on-demand strong consistency. By providing observable atomic consistency, OACP avoids the anomalies of related protocols. We provide a distributed implementation of OACP based on Akka, a widely-used actor-based middleware. Our experimental evaluation shows that OACP can reduce coordination overhead compared to other protocols providing atomic consistency. Our results also suggest that OACP increases availability through mergeable data types and provides acceptable latency for achieving strong consistency.

Usefulness of cerebrospinal fluid bacterial antigen tests

Neck dissection is the standard staging procedure to ascertain the pathologic status of cervical lymph nodes in patients with oral cavity squamous cell carcinoma (OSCC), but it results in multiple morbidities. To examine outcomes of patients with OSCC who underwent sentinel node biopsy (SNB) as the sole neck staging procedure. Retrospective review of patients who underwent SNB during the period 2005 through 2011. National Cancer Institute–designated comprehensive cancer center. Thirty-eight patients with clinically T1 or T2N0 OSCC. Preoperative lymphoscintigraphy with intraoperative gamma probe localization was used. Sentinel lymph nodes were serially sectioned, formalin fixed, and examined at 3 levels. All patients with positive SNB results underwent neck dissection, and the patients with negative SNB results were observed clinically. Sensitivity and predictive value of SNB, recurrence rates, and disease-specific survival rates. There were 18 T1 and 20 T2 tumors. Five patients had positive SNB results, of whom 3 had additional positive nodes on subsequent neck dissection. Two of 33 patients with negative SNB results developed a regional recurrence. The sensitivity and negative predictive value for staging the neck with SNB alone were 71% (5 of 7) and 94% (31 of 33), respectively. Mean follow-up was 31 months. The mean disease-free survival duration for patients with positive and negative SNB results was 30 and 65 months, respectively (P = .08). The disease-specific survival rate for patients with positive and negative SNB results was 80% and 91%, respectively. There was no significant difference in disease-specific survival between patients with true-negative and false-negative SNB results (34 vs 44 months; P = .38). The majority of patients with positive results on SNB had additional positive nodes on neck dissection. A low rate of isolated neck recurrence was found in patients with negative results on SNB. Individuals with negative results on SNB exhibited better overall and disease-specific survival than those with positive results.

Modern large-scale distributed systems often rely on eventually consistent replicated stores, which achieve scalability in exchange for providing weak semantic guarantees. To compensate for this weakness, researchers have proposed various abstractions for programming on eventual consistency, such as replicated data types for resolving conflicting updates at different replicas and weak forms of transactions for maintaining relationships among objects. However, the subtle semantics of these abstractions makes using them correctly far from trivial.To address this challenge, we propose composite replicated data types, which formalise a common way of organising applications on top of eventually consistent stores. Similarly to an abstract data type, a composite data type encapsulates objects of replicated data types and operations used to access them, implemented using transactions.We develop a method for reasoning about programs with composite data types that reflects their modularity: the method allows abstracting away the internals of composite data type implementations when reasoning about their clients. We express the method as a denotational semantics for a programming language with composite data types. We demonstrate the effectiveness of our semantics by applying it to verify subtle data type examples and prove that it is sound and complete with respect to a standard non-compositional semantics.KeywordsData TypeDenotational SemanticAtomic BlockStrict Partial OrderPrimitive ObjectThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The flow cytometric crossmatch is a technique that is increasingly being used by clinical transplant laboratories. In this multicenter study by the British Society for Histocompatibility and Immunogenetics Flow Cytometry Group, a series of crossmatches were carried out to determine whether different centers obtained same results when performing the same crossmatch. There was greater than 80% agreement among participating laboratories on the results of 35/54 tests. There was no clear agreement in the remaining 20 cases. Quantitative analysis, estimating the number of cell-bound fluorescein molecules, demonstrated that differences in the criteria used by each center to define a positive crossmatch were responsible for some discordant results. When applied, definition of positivity based on the molecules of fluorescein increased concordance from 57.5% to 81.4%.l. These results suggest that a criterion for the interpretation of results based on quantitative analysis of bound antibody may be more reliable than methods in current routine use.

A Critique on Contemporary Reporting of Likelihood Ratios in Test Power Analysis

Risk of Oral and Upper Gastrointestinal Cancers in Persons With Positive Results From a Fecal Immunochemical Test in a Colorectal Cancer Screening Program

We present our framework DKVF that enables one to quickly prototype and evaluate new protocols for key-value stores and compare them with existing protocols based on selected benchmarks. Due to limitations of CAP theorem, new protocols must be developed that achieve the desired trade-off between consistency and availability for the given application at hand. Hence, both academic and industrial communities focus on developing new protocols that identify a different (and hopefully better in one or more aspect) point on this trade-off curve. While these protocols are often based on a simple intuition, evaluating them to ensure that they indeed provide increased availability, consistency, or performance is a tedious task. Our framework, DKVF, enables one to quickly prototype a new protocol as well as identify how it performs compared to existing protocols for pre-specified benchmarks. Our framework relies on YCSB (Yahoo! Cloud Servicing Benchmark) for benchmarking. We demonstrate DKVF by implementing four existing protocols --eventual consistency, COPS, GentleRain and CausalSpartan-- with it. We compare the performance of these protocols against different loading conditions. We find that the performance is similar to our implementation of these protocols from scratch. And, the comparison of these protocols is consistent with what has been reported in the literature. Moreover, implementation of these protocols was much more natural as we only needed to translate the pseudocode into Java (and add the necessary error handling). Hence, it was possible to achieve this in just 1-2 days per protocol. Finally, our framework is extensible. It is possible to replace individual components in the framework (e.g., the storage component).

Autoimmune response to Chlamydia trachomatis infection and in vitro fertilization outcome