Stale Data Research Articles

US school ranking names no winners

Graduate programmes are assessed and measured, but stale data could reduce impact of long-aivaited report.

Improving flash translation layer performance by supporting large superblocks

A Flash Translation Layer (FTL) provides a block device interface on top of flash memory to support existing disk based file systems. Due to the erase-before-write feature of flash memory, an FTL usually performs out-of-place updates and uses a garbage collection procedure to reclaim stale data. Superblock FTL is one of the well-known FTLs, which achieves good performance in terms of both garbage collection overhead and RAM usage. The use of fine-grained mapping information allows a logical page within a superblock, i.e., a set of contiguous logical blocks, to be placed at any page offset of the physical blocks allocated for that superblock. In addition, the fine-grained mapping information is stored in the spare area of NAND flash memory to reduce the RAM usage. Generally, the FTL performance improves with larger superblocks. However, the spare area management approach of the FTL prevents it from supporting large superblocks. In this paper, we propose an FTL that incorporates a novel spare area management approach to enable the support of large superblocks. The simulation results from four traces show that the proposed FTL reduces the garbage collection overhead by up to 89%, when compared to the Superblock FTL. In addition, it results in shorter response time.

LazyBase

Information management applications exhibit a wide range of query performance and result freshness goals. Some applications, such as web search, require interactive performance, but may safely operate on stale data. Others, such as policy violation detection, require up-to-date results, but can tolerate relaxed performance goals. Furthermore, information processing applications must be able to ingest updates at the scale of an entire organization. In this paper, we present LazyBase, a system that allows users to trade off query performance and result freshness in order to satisfy the full range of users' goals. LazyBase breaks up data ingestion into a pipeline of operations to minimize ingest time and uses models of processing and query performance to execute user queries. Initial results with LazyBase illustrate the feasibility of the pipelined model, highlight a rich space of trade-offs between result freshness and query performance, and often outperform existing solutions in the space.

Continuity analysis of programs

We present an analysis to automatically determine if a program represents a continuous function, or equivalently, if infinitesimal changes to its inputs can only cause infinitesimal changes to its outputs. The analysis can be used to verify the robustness of programs whose inputs can have small amounts of error and uncertainty---e.g., embedded controllers processing slightly unreliable sensor data, or handheld devices using slightly stale satellite data. Continuity is a fundamental notion in mathematics. However, it is difficult to apply continuity proofs from real analysis to functions that are coded as imperative programs, especially when they use diverse data types and features such as assignments, branches, and loops. We associate data types with metric spaces as opposed to just sets of values, and continuity of typed programs is phrased in terms of these spaces. Our analysis reduces questions about continuity to verification conditions that do not refer to infinitesimal changes and can be discharged using off-the-shelf SMT solvers. Challenges arise in proving continuity of programs with branches and loops, as a small perturbation in the value of a variable often leads to divergent control-flow that can lead to large changes in values of variables. Our proof rules identify appropriate ``synchronization points'' between executions and their perturbed counterparts, and establish that values of certain variables converge back to the original results in spite of temporary divergence. We prove our analysis sound with respect to the traditional epsilon-delta definition of continuity. We demonstrate the precision of our analysis by applying it to a range of classic algorithms, including algorithms for array sorting, shortest paths in graphs, minimum spanning trees, and combinatorial optimization. A prototype implementation based on the Z3 SMT-solver is also presented.

Aging Bloom Filter with Two Active Buffers for Dynamic Sets

A Bloom filter is a simple but powerful data structure that can check membership to a static set. As Bloom filters become more popular for network applications, a membership query for a dynamic set is also required. Some network applications require high-speed processing of packets. For this purpose, Bloom filters should reside in a fast and small memory, SRAM. In this case, due to the limited memory size, stale data in the Bloom filter should be deleted to make space for new data. Namely the Bloom filter needs aging like LRU caching. In this paper, we propose a new aging scheme for Bloom filters. The proposed scheme utilizes the memory space more efficiently than double buffering, the current state of the art. We prove theoretically that the proposed scheme outperforms double buffering. We also perform experiments on real Internet traces to verify the effectiveness of the proposed scheme.

Cooperative caching for homogeneous wireless sensor networks

In many scenarios, data generated at sensor nodes in wireless sensor networks must be stored temporarily in a network for subsequent use. An individual sensor node's storage capacity is too small to hold it and may soon overflow. Therefore, we propose an effective caching strategy based on cooperation among various sensor nodes to form a larger cumulative cache. As part of the complete caching scheme, a novel token-based cache admission control scheme is devised, which ensures proximity of cached data closer to sink(s). A cache discovery mechanism to efficiently fetch a copy of data items is developed. A single-copy cache coherency scheme is developed to avoid sink from fetching stale data and also avoids undesired data replication. Further, a least utility based item-replacement policy is developed to evict data items with the least utility in case of desired replacement. Simulation results show significant improvements in network energy savings and data availability when cooperative caching is used.

Concurrent view maintenance scheme for soft real-time data warehouse systems

The data warehouse should be refreshed periodically or aperiodically in order to prevent OLAP transactions from reading extremely stale data. This refreshing process is referred to as view maintenance. There have been many researches on ways to preserve data consistency during the process of view maintenance. The simplest approach among them is to perform view maintenance in the nighttime while OLAP transactions are allowed to be active only in daytime. However, this approach cannot be applied to globalized corporations because no common idle time is available for such corporations. In this paper, we propose a new algorithm for performing view maintenance without blocking OLAP transactions. Our algorithm improves data freshness by allowing one or more maintenance transactions and OLAP transactions to be executed concurrently. We introduce the details of algorithm with some examples, present proofs of devised scheme, and then evaluate performance gains by means of simulation method.

Defending against cache consistency attacks in wireless ad hoc networks

Caching techniques can be used to reduce bandwidth consumption and data access delay in wireless ad hoc networks. When cache is used, the issue of cache consistency must be addressed, and maintaining strong cache consistency is desired in some strategic scenarios (e.g., battlefields). In these situations, the invalidation-based approach is preferred due to its low overhead. However, this approach may suffer from some security attacks. For example, malicious nodes (also called intruders) may drop, insert or modify invalidation messages to mislead receivers to use stale data or unnecessarily invalidate data that are still valid. In this paper, we first propose to employ the Invalidation Report (IR) based cache invalidation strategy to prevent intruders from dropping or modifying invalidation messages. Although digital signatures can be used to protect IRs, this has significantly high overhead in terms of computational and bandwidth overhead. To address this problem, we further propose a family of randomized grouping-based schemes for intrusion detection, damage recovery and intruder identification. Extensive analysis and simulations are performed to evaluate the proposed schemes. The results show that our solution can achieve a satisfactory level of security with low overhead.

The American Community Survey: Warmer (More Current), but Fuzzier (Less Precise) than the Decennial Census

The American Community Survey, which will replace data many planners rely on from the decennial Census long form, is finally in progress. The first nationwide data for places of 65,000 or more was released in the summer of 2006. It has several interesting implications for planning. On the one hand, more current data will eliminate many of the inaccuracies introduced by projection-based updates of stale census data. On the other, smaller sample sizes will mean we will have less precise estimates. Because the ACS will use averaged rather than point-in-time data, it will measure slightly different things than the decennial census. Finally, planners should be alert to the opportunities they will have to improve local data quality by improving the address file from which the sample is drawn.

Alternative Architectures and Protocols for Providing Strong Consistency in Dynamic Web Applications

Dynamic Web applications have gained a great deal of popularity. Improving the performance of these applications has recently attracted the attention of many researchers. One of the most important techniques proposed for this purpose is caching, which can be done at different locations and within different stages of the process of generating a dynamic Web page. Most of the caching schemes proposed in literature are lenient about the issue of consistency; they assume that users can tolerate receiving stale data. However, an important class of dynamic Web applications are those in which users always expect to get the freshest data available. Any caching scheme has to incur a significant overhead to be able to provide this level of consistency (i.e., strong consistency); the overhead may be so much that it neutralizes the benefits of caching. In this paper, three alternative architectures are investigated for dynamic Web applications that require strong consistency. A proxy caching scheme is designed and implemented, which performs caching at the level of database queries. This caching system is used in one of the alternative architectures. The performance experiments show that, despite the high overhead of providing strong consistency in database caching, this technique can improve the performance of dynamic Web applications, especially when there is a long network latency between clients and the (origin) server.

Optimal Synchronization Policies for Data Warehouses

The notion of a data warehouse for integrating operational data into a single repository is rapidly becoming popular in modern organizations. An important issue in this context is how often one should synchronize the data warehouse to reflect the changes in the constituent operational data sources. If the synchronization is performed very frequently, the associated cost might be quite high, although the data warehouse would only have a small amount of stale data. On the other hand, if the data warehouse is synchronized infrequently, it might result in costly errors in business decisions arising from the stale data. This paper examines the trade-off between the synchronization and staleness costs and derives the optimal synchronization frequency.

B-tree concurrency control and recovery in page-server database systems

We develop new algorithms for the management of transactions in a page-shipping client-server database system in which the physical database is organized as a sparse B-tree index. Our starvation-free fine-grained locking protocol combines adaptive callbacks with key-range locking and guarantees repeatable-read-level isolation (i.e., serializability) for transactions containing any number of record insertions, record deletions, and key-range scans. Partial and total rollbacks of client transactions are performed by the client. Each structure modification such as a page split or merge is defined as an atomic action that affects only two levels of the B-tree and is logged using a single redo-only log record, so that the modification never needs to be undone during transaction rollback or restart recovery. The steal-and-no-force buffering policy is applied by the server when flushing updated pages onto disk and by the clients when shipping updated data pages to the server, while pages involved in a structure modification are forced to the server when the modification is finished. The server performs the restart recovery from client and system failures using an ARIES/CSA-based recovery protocol. Our algorithms avoid accessing stale data but allow a data page to be updated by one client transaction and read by many other client transactions simultaneously, and updates may migrate from a data page to another in structure modifications caused by other transactions while the updating transaction is still active.

Toward Better Risk Forecasts

Volatility forecasts crucial to many financial applications usually assume implicitly that the frequency of the data should match the forecast horizon; portfolio managers typically rely on risk models estimated using monthly data to produce monthly volatility forecasts, for example. For longer–term forecasts, this practice has two drawbacks: Volatility estimates can be based on stale data; and return events occurring within long sampling intervals are obscured, confounding estimation. Monthly volatility risk measures constructed using higher–frequency data seem to be more robust than those using low–frequency data. Microstructure effects can explain the differences in estimates.

Building Scalable Electronic Market Places Using HyperQuery-Based Distributed Query Processing

Flexible distributed query processing capabilities are an important prerequisite for building scalable Internet applications, such as electronic Business-to-Business (B2B) market places. Architecting an electronic market place in a conventional data warehouse-like approach by integrating all the data from all participating enterprises in one centralized repository incurs severe problems: stale data, data security threats, administration overhead, inflexibility during query processing, etc. In this paper we present a new framework for dynamic distributed query processing based on so-called HyperQueries which are essentially query evaluation sub-plans "sitting behind" hyperlinks. Our approach facilitates the pre-materialization of static data at the market place whereas the dynamic data remains at the data sources. In contrast to traditional data integration systems, our approach executes essential (dynamic) parts of the data-integrating views at the data sources. The other, more static parts of the data are integrated a priori at the central portal, e.g., the market place. The portal serves as an intermediary between clients and data providers which execute their sub-queries referenced via hyperlinks. The hyperlinks are embedded as attribute values within data objects of the intermediary's database. Retrieving such a virtual object will execute the referenced HyperQuery in order to materialize the missing data. We illustrate the flexibility of this distributed query processing architecture in the context of B2B electronic market places with an example derived from the car manufacturing industry. Based on these HyperQueries, we propose a reference architecture for building scalable and dynamic electronic market places. All administrative tasks in such a distributed B2B market place are modeled as Web services and are initiated decentrally by the participants. Thus, sensitive data remains under the full control of the data providers. We describe optimization and implementation issues to obtain an efficient and highly flexible data integration platform for electronic market places. All proposed techniques have been fully implemented in our QueryFlow prototype system which served as the platform for our performance evaluation.

Data Cache Prefetching Using a Global History Buffer

Over the past couple of decades, trends in both microarchitecture and underlying semiconductor technology have significantly reduced microprocessor clock periods. These trends have significantly increased relative main-memory latencies as measured in processor clock cycles. To avoid large performance losses caused by long memory access delays, microprocessors rely heavily on a hierarchy of cache memories. But cache memories are not always effective, either because they are not large enough to hold a program's working set, or because memory access patterns don't exhibit behavior that matches a cache memory's demand-driven, line-structured organization. To partially overcome cache memories' limitations, we organize data cache prefetch information in a new way, a GHB (global history buffer) supports existing prefetch algorithms more effectively than conventional prefetch tables. It reduces stale table data, improving accuracy and reducing memory traffic. It contains a more complete picture of cache miss history and is smaller than conventional tables.

Programming with transactional coherence and consistency (TCC)

Transactional Coherence and Consistency (TCC) offers a way to simplify parallel programming by executing all code within transactions. In TCC systems, transactions serve as the fundamental unit of parallel work, communication and coherence. As each transaction completes, it writes all of its newly produced state to shared memory atomically, while restarting other processors that have speculatively read stale data. With this mechanism, a TCC-based system automatically handles data synchronization correctly, without programmer intervention. To gain the benefits of TCC, programs must be decomposed into transactions. We describe two basic programming language constructs for decomposing programs into transactions, a loop conversion syntax and a general transaction-forking mechanism. With these constructs, writing correct parallel programs requires only small, incremental changes to correct sequential programs. The performance of these programs may then easily be optimized, based on feedback from real program execution, using a few simple techniques.

Programming with transactional coherence and consistency (TCC)

Transactional Coherence and Consistency (TCC) offers a way to simplify parallel programming by executing all code within transactions. In TCC systems, transactions serve as the fundamental unit of parallel work, communication and coherence. As each transaction completes, it writes all of its newly produced state to shared memory atomically, while restarting other processors that have speculatively read stale data. With this mechanism, a TCC-based system automatically handles data synchronization correctly, without programmer intervention. To gain the benefits of TCC, programs must be decomposed into transactions. We describe two basic programming language constructs for decomposing programs into transactions, a loop conversion syntax and a general transaction-forking mechanism. With these constructs, writing correct parallel programs requires only small, incremental changes to correct sequential programs. The performance of these programs may then easily be optimized, based on feedback from real program execution, using a few simple techniques.

Programming with transactional coherence and consistency (TCC)

Transactional Coherence and Consistency (TCC) offers a way to simplify parallel programming by executing all code within transactions. In TCC systems, transactions serve as the fundamental unit of parallel work, communication and coherence. As each transaction completes, it writes all of its newly produced state to shared memory atomically, while restarting other processors that have speculatively read stale data. With this mechanism, a TCC-based system automatically handles data synchronization correctly, without programmer intervention. To gain the benefits of TCC, programs must be decomposed into transactions. We describe two basic programming language constructs for decomposing programs into transactions, a loop conversion syntax and a general transaction-forking mechanism. With these constructs, writing correct parallel programs requires only small, incremental changes to correct sequential programs. The performance of these programs may then easily be optimized, based on feedback from real program execution, using a few simple techniques.

Conflict Order-Based View Refreshment Scheme for Transaction Management in Data Warehouse Environment

Online analytical processing transactions (OLAP for short) are executed in integrated a data warehouse (DWH for short). DWH should be refreshed by recent value of data in source databases in order to prevent OLAP transactions from reading excessively stale data. This refreshing process is referred to as view maintenance. There have been many researches on ways to guarantee data consistency during the process of view maintenance. The most commonly used approach among them is to execute updated transactions in the nighttime while OLAP transactions are executed only in daytime. It is, however, meaningless to argue about absolute nighttime for a specific corporation which is globalized. We proposed so called COB algorithm for view maintenance in DWH environment. COB can improve data freshness by supporting concurrent execution of one or more update transactions. COB can, moreover, reduce the cost of space management by maintaining only one additional version during the process of view maintenance.

Lifetime packet discard for efficient real-time transport over cellular links

Mobile cellular users often experience significant delay jitter that undermines quality of real-time applications. Delay jitter can cause unnecessary delivery of stale packets with passed playback deadline and duplicate packets retransmitted by the end host after experiencing a timeout. With Lifetime Packet Discard (LPD) a flow adaptive link can tailor the trade-off between the maximum delay jitter and reliability if quality of service requirements of a flow are known. We propose using an IP option to communicate the flow requirements to the link layer. The packet lifetime is set to the minimum of the data lifetime determined by application and the retransmit timeout value determined by the transport protocol if selective reliability is supported. For congestion-responsive flows, the link transmits only headers of expired packets to prevent unnecessary triggering of end-to-end congestion control. Our simulations show that LPD is efficient in reducing stale data delivery and increases the number of packets delivered in time for real-time flows. For semi-reliable flows throughput and goodput are improved because duplicate packet delivery is prevented.