Skip List Research Articles

Accurate Location in Batch Dynamic Provable Data Possession

The paper focused on the construction of an efficient DPDP for public audit. We improved the existing proof of storage model by manipulating authenticated skip list structure for authentication. We further explored embedded MHT structure helping our scheme to accurate locate the incorrect part in batch auditing. Extensive security and performance evaluation showed the proposed model is highly efficient and a nice trade-off between robust construction and storage cost.

Authenticated data structures, generically

An authenticated data structure (ADS) is a data structure whose operations can be carried out by an untrusted prover , the results of which a verifier can efficiently check as authentic. This is done by having the prover produce a compact proof that the verifier can check along with each operation's result. ADSs thus support outsourcing data maintenance and processing tasks to untrusted servers without loss of integrity. Past work on ADSs has focused on particular data structures (or limited classes of data structures), one at a time, often with support only for particular operations. This paper presents a generic method, using a simple extension to a ML-like functional programming language we call λ• (lambda-auth), with which one can program authenticated operations over any data structure defined by standard type constructors, including recursive types, sums, and products. The programmer writes the data structure largely as usual and it is compiled to code to be run by the prover and verifier. Using a formalization of λ• we prove that all well-typed λ• programs result in code that is secure under the standard cryptographic assumption of collision-resistant hash functions. We have implemented λ• as an extension to the OCaml compiler, and have used it to produce authenticated versions of many interesting data structures including binary search trees, red-black+ trees, skip lists, and more. Performance experiments show that our approach is efficient, giving up little compared to the hand-optimized data structures developed previously.

Data integrity verification scheme with designated verifiers for dynamic outsourced databases

ABSTRACTIn order to verify the integrity of the query results in outsourced database model, we first present a new authenticated data structure by combining skip lists with polynomial functions and accumulators and then construct a data integrity verification scheme for range query. Our scheme can verify the integrity of the query results with constant time verification and constant size proof and support dynamic maintenance with constant computational cost. Compared with that of previous schemes, the communication cost in our scheme is independent of the size of the authenticated dataset and is lower. When dataset size gets larger enough, our scheme outperforms previous schemes in terms of the computational cost and verification cost. In addition, we introduce the designated verifier idea into our scheme, so our scheme achieves the result that none but subscribers can verify the integrity of the query results and then meets the practical demands such as the payment service. The proof of security and analysis of experiment performance show that our scheme is secure and efficient for larger outsourced databases. Copyright © 2013 John Wiley & Sons, Ltd.

Supporting Lock-Free Composition of Concurrent Data Objects: Moving Data between Containers

Lock-free data objects offer several advantages over their blocking counterparts, such as being immune to deadlocks, priority inversion, and convoying. They have also been shown to work well in practice. However, composing the operations they provide into larger atomic operations, while still guaranteeing efficiency and lock-freedom, is a challenging algorithmic task. We present a lock-free methodology for composing a wide variety of concurrent linearizable objects together by unifying their linearization points. This makes it possible to relatively easily introduce atomic lock-free move operations to a wide range of concurrent lock-free containers. This move operation allows data to be transferred from one container to another, in a lock-free way, without blocking any of the operations supported by the original container. For a data object to be suitable for composition using our methodology it needs to fulfill a set of requirements. These requirement are, however, generic enough to be fulfilled by a large set of objects. To show this we have performed case studies on six commonly used lock-free objects (a stack, a queue, a skip list, a deque, a doubly linked list and a hash table) to demonstrate the general applicability of the methodology. We also show that the operations originally supported by the data objects keep their performance behavior under our methodology.

Distributed deterministic 1–2 skip list for peer-to-peer system

Data management in the peer-to-peer system is a challenging task due to the random distribution of data among several participating peers. Efficient data structures like distributed hash tables (DHT) and its variants are designed and implemented to reduce the complexity of data management in such environment. However, DHT has its limitations in supporting range queries and its variants like distributed segment trees often perform poorly when the number of peers is high. Further, distributed lists and distributed balanced trees require significant amount of time for stabilizing after a new peer joins or a peer leaves. In this paper, a new distributed data structure called deterministic 1–2 skip list is introduced as an alternate solution for data management in the peer-to-peer systems. A deterministic skip list can be viewed as an alternate of a balanced tree, where the semantic locality of each key is preserved. Thus it can support the range queries as well as the single shot queries. This paper proposes three main operations on this data structure - searching data based on keys, insertion when a new peer joins, and deletion when a peer leaves. The correctness of the proposed operations are analyzed using theoretical arguments and mathematical proofs. The proposed scheme is simulated using NS-2.34 network simulator, and the efficiency of the scheme has been compared with DHT, DST, distributed list and distributed tree based data management.

An Skip List Based Authentication Scheme for Encrypted Outsourced Database

An Skip List Based Authentication Scheme for Encrypted Outsourced Database

Corona: A stabilizing deterministic message-passing skip list

We present Corona, a deterministic self-stabilizing algorithm for skip list construction in structured overlay networks. Corona operates in the low-atomicity message-passing asynchronous system model. Corona requires constant process memory space for its operation and, therefore, scales well. We prove the general necessary conditions limiting the initial states from which a self-stabilizing structured overlay network in a message-passing system can be constructed. The conditions require that initial state information has to form a weakly connected graph and it should only contain identifiers that are present in the system. We formally describe Corona and rigorously prove that it stabilizes from an arbitrary initial state subject to the necessary conditions. We extend Corona to construct a skip graph.

Window-based greedy contention management for transactional memory: theory and practice

We consider greedy contention managers for transactional memory for M × N execution windows of transactions with M threads and N transactions per thread. We present, formally analyze, and experimentally evaluate three new randomized greedy contention management algorithms for transaction windows. Assuming that each transaction has duration τ and conflicts with at most C other transactions inside the window, the first algorithm Offline-Greedy produces a schedule of length O(τ· (C + N· log(MN))) with high probability. The offline algorithm depends on knowing the conflict graph which evolves while the execution of the transactions progresses. The second algorithm Online-Greedy produces a schedule of length that is only a logarithmic factor worse than Offline-Greedy, but does not require knowledge of the conflict graph. The third algorithm Adaptive-Greedy is the adaptive version of the previous algorithms which produces a schedule of length asymptotically the same as with online algorithm by adaptively guessing the value of C. All of the algorithms exhibit competitive ratio very close to O(s), where s is the number of shared resources, and at the same time, our algorithms provide new non-trivial tradeoffs for greedy transaction scheduling that parameterize window sizes and transaction conflicts within the execution window. We evaluate these window-based algorithms experimentally using the sorted link list, red-black tree, skip list, and vacation benchmarks. The evaluation results confirm their benefits in practical performance throughput and other metrics such as aborts per commit ratio and execution time overhead, along with the non-trivial provable properties of the algorithms.

Tiara: A self-stabilizing deterministic skip list and skip graph

We present Tiara—a self-stabilizing peer-to-peer network maintenance algorithm. Tiara is truly deterministic which allows it to achieve exact performance bounds. Tiara allows logarithmic searches and topology updates. It is based on a novel sparse 0–1 skip list. We then describe its extension to a ringed structure and to a skip-graph.

Skip lift: A probabilistic alternative to red–black trees

We present the Skip lift, a randomized dictionary data structure inspired by the skip list [Pughʼ90, Comm. of the ACM]. Similar to the skip list, the skip lift has the finger search property: given a pointer to an arbitrary element f, searching for an element x takes expected O(logδ) time where δ is the rank distance between the elements x and f. The skip lift uses nodes of O(1) worst-case size (for a total of O(n) worst-case space usage) and it is one of the few efficient dictionary data structures that performs an O(1) worst-case number of structural changes (pointers/fields modifications) during an update operation. Given a pointer to the element to be removed from the skip lift the deletion operation takes O(1) worst-case time.

Complex window query support for monitoring streaming data in wireless body area networks

Wireless Body Area Networks (WBAN) combined with various body sensors and personal electronic devices are emerging as a new platform for medical applications. This paper proposes a real-time monitoring system with complex window query support for enabling advanced monitoring on streaming data that is produced by the body sensors and transmitted through the WBAN. The main techniques developed are the caching for multi-source data stream support, complex range expression processing with interval skip lists, and historic data processing for sliding window condition monitoring support. The condition monitoring query language and efficient processing mechanisms to support the complex monitoring are presented. Experimental results show that the proposed approach provides a wider spectrum of monitoring functions as well as enhancing the scalability and efficiency of the monitoring.

Consecutive records in geometrically distributed words

Words \({a_1a_2\ldots a_n}\) with independent letters ak taken from the set of natural numbers, and a weight (probability) attached via the geometric distribution pqi−1 (p + q = 1) are considered. The parameter \({{\mathcal K}(a_1a_2\ldots a_n)}\) , (the number of weak consecutive records), has proved to be essential in the analysis of a skip list structure. Related to it is the (new) parameter \({{\mathcal M}}\) , i.e., the largest consecutive record in a random word of length n. Exact and asymptotic formulae are derived for the expectation and the variance.

Column-oriented storage techniques for MapReduce

Users of MapReduce often run into performance problems when they scale up their workloads. Many of the problems they encounter can be overcome by applying techniques learned from over three decades of research on parallel DBMSs. However, translating these techniques to a Map-Reduce implementation such as Hadoop presents unique challenges that can lead to new design choices. This paper describes how column-oriented storage techniques can be incorporated in Hadoop in a way that preserves its popular programming APIs. We show that simply using binary storage formats in Hadoop can provide a 3x performance boost over the naive use of text files. We then introduce a column-oriented storage format that is compatible with the replication and scheduling constraints of Hadoop and show that it can speed up MapReduce jobs on real workloads by an order of magnitude. We also show that dealing with complex column types such as arrays, maps, and nested records, which are common in MapReduce jobs, can incur significant CPU overhead. Finally, we introduce a novel skip list column format and lazy record construction strategy that avoids deserializing unwanted records to provide an additional 1.5x performance boost. Experiments on a real intranet crawl are used to show that our column-oriented storage techniques can improve the performance of the map phase in Hadoop by as much as two orders of magnitude.

How often do we reject a superior value?

Words a 1 a 2…a n with independent letters ak taken from the set of natural numbers, and a weight (probability) attached via the geometric distribution pq i–1 (p + q = 1) are considered. A consecutive record (motivated by the analysis of a skip list structure) can only advance from k to k + 1 when k is the current consecutive record, and the value k + 1 is seen when scanning the word from left to right. Some larger (= superior) values are therefore ignored (= rejected). We investigate the number of these rejected superior values. Further, we study the probability that there is a single consecutive maximum and show that (apart from fluctuations) it tends to a constant.

How often do we reject a superior value? (Extended abstract)

Words $a_1 a_2 \ldots a_n$ with independent letters $a_k$ taken from the set of natural numbers, and a weight (probability) attached via the geometric distribution $pq^{i-1}(p+q=1)$ are considered. A consecutive record (motivated by the analysis of a skip list structure) can only advance from $k$ to $k+1$, thus ignoring perhaps some larger (=superior) values. We investigate the number of these rejected superior values. Further, we study the probability that there is a single consecutive maximum and show that (apart from fluctuations) it tends to a constant. On considère des mots $a_1a_2 \ldots a_n$ formés de lettres à valeurs entières, tirées de façon indépendante avec une distribution géométrique $pq^{i-1}(p+q=1)$. Un record $k+1$ est dit consécutif si la lettre précédente est $k$. La notion est motivée par des considérations algorithmiques. Les autres records sont rejetés. Nous étudions le nombre de records rejetés. Nous étudions aussi la probabilité qu'il y ait un seul maximum consécutif, et montrons qu'elle converge vers une constante, à certaines fluctuations près.

스킵리스트를 이용한 인터넷 토론 게시판 댓글 관리

최근 웹 블로그나 인터넷 게시판과 같은 가상 커뮤니티가 활발히 사용됨에 따라 댓글을 통해 자신의 의견을 적극적으로 나타내고자 하는 이용자들이 점점 증가하고 있는 추세다. 실제로 댓글 활동이 활발한 인터넷 토론 게시판에서 수천 개의 댓글이 달린 게시물도 어렵지 않게 찾아볼 수 있다. 대부분의 웹 블로그나 인터넷 게시판에서는 댓글이 작성된 시간에 따라 목록 형태로만 제공되고 있을 뿐 기본적인 검색 기능조차도 지원되지 않고 있다. 본 논문에서는 인터넷 토론 게시판의 댓글 분석을 통해 댓글 작성자의 분포가 거듭제곱 법칙을 따르는 것을 밝혔다. 그리고 이러한 댓글의 통계적 특성을 반영하는 스킵리스트 기반의 댓글 검색 구조를 제안한다. 제안 방법의 주안점 댓글 작성자들의 확률적 특성을 데이터 구조에 반영하는 것이다. 실험을 통해 제안 방법이 B-트리나 일반적인 스킵리스트의 이론적인 계산 복잡도인 logN에 비해 더 빠른 검색을 수행할 수 있음을 보인다. In recent years, the number of users who are actively express their opinions about Internet articles is more and more growing up, as the use of cyber community such as weblog or Internet discussion board increases. In fact, it is not difficult to find an article with hundreds of comments in famous Internet discussion boards. Most of the weblogs or Internet discussion boards present comments in the form of list and do not yet support even the basic operation such as searching comments. In this paper, we analysed large sets of comments in Internet discussion board named AGORA. It was found that from the result that the distribution of comment writers follows power-law. So we suppose a new search structure of comments using skip lists. The main idea of our approach is to reflect the probabilistic distribution properties of the commenters following the power-law to the data structure. Our empirical results show that the proposed method performs more efficient in searching the nodes with fewer number of comparison operations than logN, which is the theoretical time complexity of general indexed structure such as B-trees or typical skip lists.

A practical concurrent binary search tree

We propose a concurrent relaxed balance AVL tree algorithm that is fast, scales well, and tolerates contention. It is based on optimistic techniques adapted from software transactional memory, but takes advantage of specific knowledge of the the algorithm to reduce overheads and avoid unnecessary retries. We extend our algorithm with a fast linearizable clone operation, which can be used for consistent iteration of the tree. Experimental evidence shows that our algorithm outperforms a highly tuned concurrent skip list for many access patterns, with an average of 39% higher single-threaded throughput and 32% higher multi-threaded throughput over a range of contention levels and operation mixes.

A skip-list approach for efficiently processing forecasting queries

Time series data is common in many settings including scientific and financial applications. In these applications, the amount of data is often very large. We seek to support prediction queries over time series data. Prediction relies on model building which can be too expensive to be practical if it is based on a large number of data points. We propose to use statistical tests of hypotheses to choose a proper subset of data points to use for a given prediction query interval. This involves two steps: choosing a proper history length and choosing the number of data points to use within this history. Further, we use an I/O conscious skip list data structure to provide samples of the original data set. Based on the statistics collected for a query workload, which we model as a probability mass function (PMF) over query intervals, we devise a randomized algorithm that selects a set of pre-built models (PM's) to construct, subject to some maintenance cost constraint when there are updates. Given this set of PM's, we discuss interesting query processing strategies for not only point queries, but also range, aggregation, and JOIN queries. We conduct a comprehensive empirical study on real world datasets to verify the effectiveness of our approaches and algorithms.

QUISIS: Interval Skip List를 활용한 질의 색인 기법

Due to the proliferation of the Internet and intranet, new application domains such as stream data processing have emerged. Stream data is real-timely and continuously generated. In stream data environments, a lot of queries are registered, and then, the arrived data item is evaluated by registered queries. Thus, to accelerate the query performance, diverse continuous query index schemes have been proposed for stream data processing systems. In this paper, we focus on the query index technique for stream data. In general, a stream query contains the range condition. Thus, by using range conditions, the queries can be indexed. In this paper, we propose an efficient query index scheme, called QUISIS, using a modified Interval Skip Lists to accelerate search time. QUISIS utilizes a locality where a value which will arrive in near future is similar to the current value. Through the experimental study, we show the efficiency of our proposed method.

SKIP QUADTREES: DYNAMIC DATA STRUCTURES FOR MULTIDIMENSIONAL POINT SETS

We present a new multi-dimensional data structure, which we call the skip quadtree (for point data in R2) or the skip octree (for point data in Rd, with constant d > 2). Our data structure combines the best features of two well-known data structures, in that it has the well-defined “box”-shaped regions of region quadtrees and the logarithmic-height search and update hierarchical structure of skip lists. Indeed, the bottom level of our structure is exactly a region quadtree (or octree for higher dimensional data). We describe efficient algorithms for inserting and deleting points in a skip quadtree, as well as fast methods for performing point location, approximate range, and approximate nearest neighbor queries.