File Requests Research Articles

Interacting branching processes and linear file-sharing networks

File-sharing networks are distributed systems used to disseminate files among nodes of a communication network. The general simple principle of these systems is that once a node has retrieved a file, it may become a server for this file. In this paper, the capacity of these networks is analyzed with a stochastic model when there is a constant flow of incoming requests for a given file. It is shown that the problem can be solved by analyzing the asymptotic behavior of a class of interacting branching processes. Several results of independent interest concerning these branching processes are derived and then used to study the file-sharing systems.

Interacting branching processes and linear file-sharing networks

File-sharing networks are distributed systems used to disseminate files among nodes of a communication network. The general simple principle of these systems is that once a node has retrieved a file, it may become a server for this file. In this paper, the capacity of these networks is analyzed with a stochastic model when there is a constant flow of incoming requests for a given file. It is shown that the problem can be solved by analyzing the asymptotic behavior of a class of interacting branching processes. Several results of independent interest concerning these branching processes are derived and then used to study the file-sharing systems.

Corrigendum to “Wikipedia workload analysis for decentralized hosting” [Computer Networks 53 (11) (2009) 1830–1845

The authors would like to point out an error in article [1]. The paper states in Section 3 that there were 20.6 billion requests in the studied period, but the correct number is 25.6 billion requests. The difference corresponds to media file (image) requests that were not considered originally. This results in some values reported in Tables 1 and 2 and parts of the text being incorrect. Also note that the read/save ratio column in Table 2 had numbers not corresponding to the studied period, but to a previous data set. The corrections to the text are as follows:

Modeling content availability in peer-to-peer swarming systems

BitTorrent, the immensely popular file swarming system, has a fundamental problem: availability. Swarming systems are peer-to-peer systems where users simultaneously interested in the same content cooperate with each other. Although swarming in BitTorrent scales impressively to tolerate massive flash crowds for popular content, it does little to disseminate unpopular content as they are limited by the availability of a seed or publisher. The extent of publisher unavailability is severe, e.g., our measurement study shows that 40% of the swarms have no publishers available more than 50% of the time. In this paper we develop a model to quantify content availability in BitTorrent. An M/G/∞ queueing system is used to capture BitTorrent’s scale-free property, i.e., more peers bring in more capacity to the system. The busy period of the queue captures an uninterrupted duration for which the content is available. The busy period increases exponentially with the arrival rate of customers as well as the time spent by each customer in the system. Armed with this model, we ask how content availability can be improved. Our model suggests an intriguing answer, namely, bundling builds availability in BitTorrent. Bundling is a growing trend today among BitTorrent publishers; instead of disseminating individual files via isolated swarms, a publisher packages a number of related files and disseminates it via a single larger swarm. To appreciate why bundling improves content availability, consider a bundle of K files. The popularity of the bundle is roughly K times the popularity of an individual file as a peer requesting any file requests the entire bundle. The size of the bundle is roughly K times the size of an individual file, so each peer stays K times as long in the system. Our model suggests that the busy period of the bundled swarm is a factor e ) larger than that of an individual swarm. If the busy period lasts until the publisher reappears, the content will be available throughout. In related work, [2] modeled the performance of peer-topeer systems. In contrast, we present a model for availability. [3] points out that peers who want a file may have to wait indefinitely to obtain certain chunks that are not available, a problem referred to as the blocked leecher problem (BLP). Here we show that bundling is one solution to mitigate this availability problem.

On-line algorithm of loose competitive caching

In response to a sequence of requests for files, where each file has a specified size and retrieval cost, the total size of each file in cache was maintained at some specified k so as to minimize the total retrieval cost. A simple deterministic on-line algorithm was proposed. In this algorithm, many well-known paging and weighted caching strategies were generalized, and the retrieval cost was either insignificant or at most a constant (independent of k) times the optimum value for most k. This helps explain why competition ratios of many on-line paging algorithms have been typically observed to be constant in practice.

Self-Adaptive Load Balancing Method in Structured P2P Protocol

PDF HTML阅读 XML下载 导出引用 引用提醒 一种结构化P2P 协议中的自适应负载均衡方法 DOI: 作者: 作者单位: 作者简介: 通讯作者: 中图分类号: 基金项目: Supported by the National Natural Science Foundation of China under Grant No.60673156 (国家自然科学基金); the Key Project ofthe Ministry of Education of China under Grant No.105129 (教育部科学技术研究重点项目) Self-Adaptive Load Balancing Method in Structured P2P Protocol Author: Affiliation: Fund Project: 摘要 | 图/表 | 访问统计 | 参考文献 | 相似文献 | 引证文献 | 资源附件 | 文章评论 摘要:提出一种自适应负载均衡方法,方法采用一种被动式结点负载统计方法生成局部负载视图;一种文件访问统计方法生成局部文件访问视图;当系统内结点负载存在差异时,重载结点把指向自身的逻辑链路迁移至指向局部负载视图中的轻载结点,通过减小重载结点入度和增加轻载结点入度来减小结点间负载差异;当结点的请求负载较高时,通过局部文件访问视图计算需要缓存的热点文件及目标结点,降低承载热点文件的结点请求负载.实验结果表明,在用户查询服从Zipf 分布的环境下,自适应负载均衡方法可使系统负载达到较好的均衡;缓存方法虽然在一定程度上增加了缓存和更新开销,但在一定条件下比查询消息命中缓存节省的网络开销要小,降低了系统的整体负载. Abstract:This paper presents a self-adaptive load balancing algorithm, in which each node creates a local load distribution view with a passive load statistic method and a local file requested view with a file requested statisticmethod. When the load imbalance exists in the system, the heavily loaded node will make the logical links pointingto itself point to a lighty loaded node in its local load distribution view, with the indegree of the heavy loaded nodedecreasing and that of the lighty loaded node increasing, the load imbalance magnitude will decrease. When therequest load of the heavy loaded node is high, the node will use its local file request view to get the popular file andcache the file to corresponding target node. Results from simulation experiments indicate that the system has a good load balance under Zipf-like requests distribution if it runs the self adaptive load balancing algorithm. To someextent, caching requires some extra messages, but fewer than the cache hit messages under some condition, socaching can reduce the overall load of the system. 参考文献 相似文献 引证文献

Algorithms and performance of load-balancing with multiple hash functions in massive content distribution

We consider a two-tier content distribution system for distributing massive content, consisting of an infrastructure content distribution network (CDN) and a large number of ordinary clients. The nodes of the infrastructure network form a structured, distributed-hash-table-based (DHT) peer-to-peer (P2P) network. Each file is first placed in the CDN, and possibly, is replicated among the infrastructure nodes depending on its popularity. In such a system, it is particularly pressing to have proper load-balancing mechanisms to relieve server or network overload. The subject of the paper is on popularity-based file replication techniques within the CDN using multiple hash functions. Our strategy is to set aside a large number of hash functions. When the demand for a file exceeds the overall capacity of the current servers, a previously unused hash function is used to obtain a new node ID where the file will be replicated. The central problems are how to choose an unused hash function when replicating a file and how to choose a used hash function when requesting the file. Our solution to the file replication problem is to choose the unused hash function with the smallest index, and our solution to the file request problem is to choose a used hash function uniformly at random. Our main contribution is that we have developed a set of distributed, robust algorithms to implement the above solutions and we have evaluated their performance. In particular, we have analyzed a random binary search algorithm for file request and a random gap removal algorithm for failure recovery.

Significant Treatment: An Assessment of Issues-Programs List Availability in Missouri Radio Station Public Inspection Files

Over the past three decades, rising competition from television and other new media prompted the FCC to ease the responsibilities of radio stations to maintain extensive records of issues programming, while retaining the responsibility of stations to provide ready public access to records. This study investigated the availability of Issues-Programs Lists in nearly 100 Missouri radio station Public Files. One in five randomly sampled stations failed to produce access to Lists upon demand as required under federal law. Noncompliance themes include: Inability to access Public Files within 20 minutes of face-to-face requests; station personnel with limited or no knowledge of Files; denial of access to Files and Lists (i.e., verbal refusals, locked studio doors); and incomplete Files. The rate of noncompliance leads the researchers to urge greater efforts among Missouri broadcasters to be aware of and to train station personnel properly to respond to File requests.

Scalable On-Demand Streaming of Nonlinear Media

A conventional video file contains a single temporally-ordered sequence of video frames. Clients requesting on-demand streaming of such a file receive (all or intervals of) the same content. For popular files that receive many requests during a file playback time, scalable streaming protocols based on multicast or broadcast have been devised. Such protocols require server and network bandwidth that grow much slower than linearly with the file request rate. This paper considers ldquononlinearrdquo video content in which there are parallel sequences of frames. Clients dynamically select which branch of the video they wish to follow, sufficiently ahead of each branch point so as to allow the video to be delivered without jitter. An example might be ldquochoose-your-own-endingrdquo movies. With traditional scalable delivery architectures such as movie theaters or TV broadcasting, such personalization of the delivered video content is very difficult or impossible. It becomes feasible, in principle at least, when the video is streamed to individual clients over a network. For on-demand streaming of nonlinear media, this paper analyzes the minimal server bandwidth requirements, and proposes and evaluates practical scalable delivery protocols.

Cooperative Client-Side File Caching for MPI Applications

Client-side file caching is one of many I/O strategies adopted by today's parallel file systems that were initially designed for distributed systems. Most of these implementations treat each client independently because clients' computations are seldom related to each other in a distributed environment. However, it is misguided to apply the same assumption directly to high-performance computers where many parallel I/O operations come from a group of processes working within the same parallel application. Thus, file caching could perform more effectively if the scope of processes sharing the same file is known. In this paper, we propose a client-side file caching system for MPI applications that perform parallel I/O operations on shared files. In our design, an I/O thread is created and runs concurrently with the main thread in each MPI process. The MPI processes that collectively open a shared file use the I/O threads to cooperate with each other to handle file requests, cache page access, and coherence control. By bringing the caching subsystem closer to the applications as a user space library, it can be incorporated into an MPI I/O implementation to increase its portability. Performance evaluations using three I/O benchmarks demonstrate a significant improvement over traditional methods that use either byte-range file locking or rely on coherent I/O provided by the file system.

Foreword

I would like to introduce this issue by telling a story. Sometime back in 1997, I wrote a paper on a new idea for improving the response times of http requests at a Web server. The idea was to schedule the HTTP requests so as to favor requests for small files, in accordance with the well-known scheduling policy Shortest Remaining Processing Time (SRPT). The paper was rejected, for many reasons, but the review that stuck in my mind was the one that said, "Why is this person writing about scheduling? Scheduling is dead." According to this reviewer, everything that would ever be known about scheduling was already described in the beautiful Theory of Scheduling book, written in 1967, by Conway, Maxwell, and Miller.

Web servers under overload

This article provides a detailed implementation study on the behavior of web serves that serve static requests where the load fluctuates over time (transient overload). Various external factors are considered, including WAN delays and losses and different client behavior models. We find that performance can be dramatically improved via a kernel-level modification to the web server to change the scheduling policy at the server from the standard FAIR (processor-sharing) scheduling to SRPT (shortest-remaining-processing-time) scheduling. We find that SRPT scheduling induces no penalties. In particular, throughput is not sacrificed and requests for long files experience only negligibly higher response times under SRPT than they did under the original FAIR scheduling.

Analysis of search and replication in unstructured peer-to-peer networks

This paper investigates the effect of the number of file replicas on search performance in unstructured peer-to-peer networks. We observe that for a search network with a random graph topology where file replicas are uniformly distributed, the hop distance to a replica of a file is logarithmic in the number of replicas. Using this observation we show that flooding-based search is optimized when the number of replicas is proportional to the file request rates. This replica distribution is also optimal for download time and since flooding has logarithmically better search time than random walk under its optimal replica distribution, we investigate the query-processing load using this distribution.

Efficient search in unstructured peer‐to‐peer networks

AbstractThe huge popularity of recent peer‐to‐peer (P2P) file sharing systems has been mainly driven by the scalability of their architectures and the flexibility of their search facilities. Such systems are usually designed as unstructured P2P networks, because they impose few constraints on topology and data placement and support highly versatile search mechanisms. A major limitation of unstructured P2P networks lies, however, in the inefficiency of their search algorithms, which are usually based on simple flooding schemes. In this paper, we propose novel mechanisms for improving search efficiency in unstructured P2P networks. Unlike other approaches, we do not rely on specialized search algorithms; instead, the peers perform local dynamic topology adaptations, based on the query traffic patterns, in order to spontaneously create communities of peers that share similar interests. The basic premise of such semantic communities is that file requests have a high probability of being fulfilled within the community they originate from, therefore increasing the search efficiency. We propose further extensions to balance the load among the peers and reduce the query traffic. Extensive simulations under realistic operating conditions substantiate that our techniques significantly improve the search efficiency and reduce the network load. Copyright © 2004 AEI

Formalizing SMART scheduling

It is well-known that policies which bias towards small job sizes or jobs with small remaining service times perform well with respect to mean response time and mean slowdown. This idea has been fundamental in many system implementations including the case of Web servers, where it has been shown that by giving priority to requests for small files, a Web server can significantly reduce mean response time and mean slowdown [1]. The heuristic has also been applied to other application areas; for example, scheduling in supercomputing centers.

Size-based scheduling to improve web performance

Is it possible to reduce the expected response time of every request at a web server, simply by changing the order in which we schedule the requests? That is the question we ask in this paper.This paper proposes a method for improving the performance of web servers servicing static HTTP requests. The idea is to give preference to requests for small files or requests with short remaining file size, in accordance with the SRPT (Shortest Remaining Processing Time) scheduling policy.The implementation is at the kernel level and involves controlling the order in which socket buffers are drained into the network. Experiments are executed both in a LAN and a WAN environment. We use the Linux operating system and the Apache and Flash web servers.Results indicate that SRPT-based scheduling of connections yields significant reductions in delay at the web server. These result in a substantial reduction in mean response time and mean slowdown for both the LAN and WAN environments. Significantly, and counter to intuition, the requests for large files are only negligibly penalized or not at all penalized as a result of SRPT-based scheduling.

Shared Files

Shared Files

On-Line File Caching

In the on-line file-caching problem problem, the input is a sequence of requests for files, given on-line (one at a time). Each file has a non-negative size and a non-negative retrieval cost. The problem is to decide which files to keep in a fixed-size cache so as to minimize the sum of the retrieval costs for files that are not in the cache when requested. The problem arises in web caching by browsers and by proxies. This paper describes a natural generalization of LRU called Landlord and gives an analysis showing that it has an optimal performance guarantee (among deterministic on-line algorithms). The paper also gives an analysis of the algorithm in a so-called ``loosely'' competitive model, showing that on a ``typical'' cache size, either the performance guarantee is O(1) or the total retrieval cost is insignificant.

Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs

We consider an architecture for a serverless distributed file system that does not assume mutual trust among the client computers. The system provides security, availability, and reliability by distributing multiple encrypted replicas of each file among the client machines. To assess the feasibility of deploying this system on an existing desktop infrastructure, we measure and analyze a large set of client machines in a commercial environment. In particular, we measure and report results on disk usage and content; file activity; and machine uptimes, lifetimes, and loads. We conclude that the measured desktop infrastructure would passably support our proposed system, providing availability on the order of one unfilled file request per user per thousand days.

The Object-Oriented Design and Performance of JAWS: A High-performance Web Server Optimized for

This paper provides two contributions to the study of high-performance object-oriented (OO) Web servers. First, it outlines the design principles and optimizations necessary to develop efficient and scalable Web servers and illustrates how we have applied these principles and optimizations to create JAWS. JAWS is a high-performance Web server that is designed to alleviate overheads incurred by existing Web servers on high-speed networks. In addition to its highly extensible OO design, it is also highly efficient, consistently outperforming existing Web servers, such as Apache, Java Server, PHTTPD, Zeus, and Netscape Enterprise, over 155 Mbps ATM networks on UNIX platforms. Second, this paper describes how we have customized the JAWS OO design to leverage advanced features of Windows NT on multi-processor platforms linked by high-speed ATM networks. The Windows NT features used in JAWS include asynchronous mechanisms for connection establishment and data transfer. Our previous benchmarking studies demonstrate that once the overhead of disk I/O is reduced to a negligible constant factor (e.g., via memory caches), the primary determinants of Web server performance are its concurrency and event dispatching strategies. Our performance results over a 155 Mbps ATM network indicate that certain Windows NT asynchronous I/O mechanisms (i.e., TransmitFile) provide superior performance for large file transfers compared with conventional synchronous multi-threaded servers. Conversely, synchronous event dispatching performed better for files less than 50 Kbytes. Thus, to provide optimal performance, a Web server design should be adaptive, i.e., choosing to use different mechanisms (such as TransmitFile) to handle requests for large files, while using alternative I/O mechanisms (such as synchronous event dispatching) on requests for small files.