Server-side Caching Research Articles

PLeveraging Django and Redis using Web Scraping

Web scraping is also known as data scraping and it is used for extracting data from sites. The software used for this may directly access the World Wide Web by using the Hypertext Transfer Protocol or by using a web browser. Over the years, due to advancements in web development and its technology, various frameworks have come in use and almost all of websites are dynamic with their content being served from CMS. This makes it tough to extract data since there is no common template for extracting data. Hence, we use RSS. Rich Site Summary is a kind of timeline allowing users and also applications to gain access to the updates on websites in a standardized, computer-readable format. This project combines the use of RSS to extract data from websites and serve users in a robust and easy way. The differentiation is that this project uses server side caching to serve users almost instantaneously without the need to perform data extraction from the requested site all over again. This is done using Redis and Django.

EFFECTS OF LOADING SPEED ON THE SITE TRAFFIC CONVERSION

The subject of this article is the impact of site loading speed to increase user conversions. The goal is to determine what is the optimal site loading speed for users and further site optimization. Tasks: identify and optimize the components that take the longest load time. The methods used are: optimization of the server part of the site; Configuring Apache and Ngix Server Using server side gzip compression Using CDNs to download popular JavaScript libraries server-side caching settings; database optimization; optimization of TCP, TLS, HTTP / 2; client side optimization. The following results are obtained. Using the methods described above, on the example of a working draft, it was possible to optimize the site loading speed by 40%. Taking statistics for the period of 7 days, with the same amount of traffic, the conversion of users to customers increased from 7% to 15%. The number of failures on the first visit to the site decreased by 30%. Conclusions. In the course of work, the correlation between the speed of loading the site and the conversion of users to customers was confirmed. By optimizing the speed of loading the site, it was possible to increase the conversion of users by 2 times and reduce the percentage of failure on the first visit to the site.

An Efficient Ring-Based Metadata Management Policy for Large-Scale Distributed File Systems

The growing size of modern file system is expected to reach EB-scale. Therefore, an efficient and scalable metadata service is critical to system performance. Distributed metadata management schemes, which use multiple metadata servers (MDS's) to store metadata, provide a highly effective approach to alleviate the workload of a single server. However, it is difficult to maintain good metadata locality and load balancing among MDS's at the same time. In this paper, we propose a novel hashing scheme called AngleCut to partition metadata namespace tree and serve large-scale distributed storage systems. AngleCut first uses a locality preserving hashing (LPH) function to project the namespace tree into linear keyspace, i.e., multiple Chord-like rings. Then we design a history-based allocation strategy to adjust the workload of MDS's dynamically. Besides, we propose a two-layer metadata cache mechanism, including server-side cache and client-side cache to provide the two stage access acceleration. Last but not least, we introduce a distributed metadata processing 2PC Protocol Based on Message Queue (2PC-MQ) to ensure data consistency. In general, our scheme preserves good metadata locality as well as maintains a high load balancing between MDS's. The theoretical proof and extensive experiments on Amazon EC2 demonstrate the superiority of AngleCut over previous literature.

Evaluating Phase Change Memory for Enterprise Storage Systems

Storage systems based on Phase Change Memory (PCM) devices are beginning to generate considerable attention in both industry and academic communities. But whether the technology in its current state will be a commercially and technically viable alternative to entrenched technologies such as flash-based SSDs remains undecided. To address this, it is important to consider PCM SSD devices not just from a device standpoint, but also from a holistic perspective. This article presents the results of our performance study of a recent all-PCM SSD prototype. The average latency for a 4KiB random read is 6.7μs, which is about 16× faster than a comparable eMLC flash SSD. The distribution of I/O response times is also much narrower than flash SSD for both reads and writes. Based on the performance measurements and real-world workload traces, we explore two typical storage use cases: tiering and caching. We report that the IOPS/$ of a tiered storage system can be improved by 12--66% and the aggregate elapsed time of a server-side caching solution can be improved by up to 35% by adding PCM. Our results show that (even at current price points) PCM storage devices show promising performance as a new component in enterprise storage systems.

TGAC Browser: visualisation solutions for big data in the genomic era

We present a new web-based genomics browser with novel rendering and annotation capabilities designed to overcome some of the shortcomings in available approaches. Most of the popular current browsers use on-the-fly server-side rendering of tracks and features which is not efficient in terms of performance or scalability. In addition, these browsers often rely on very specific library dependencies, where writing plugins or modifying existing code can be troublesome and resource expensive. We focus on functionality which in many cases is available in existing tools but with an improved and more productive user interface. Responsiveness: Server side caching and client side rendering based on JSON generated from server responses, helps to decrease the server load compared to server-side rendering browsers, and improves the user browsing experience. Availability • TGAC Browser has been released to community and is available in variety of ways, including source code. We would be pleased to help any potential users and developers interested in the project. • Information about the TGAC Browser and a demo instance please visit : http://tgac-browser.tgac.ac.uk • Follow on @TGACBrowser • TGAC Browser source code available at: https://github.com/TGAC/TGACBrowser User-friendly Browser Interaction: Traversing the reference genome, adding/removing tracks, drag and drop selections are examples of actions that are seamlessly powered by modern web browsers. In addition, other key features of the TGAC browser include merging and removing tracks and exporting sequences in FASTA format. Analysis Integration: Ability to carry out alignments using tools such BLAST is a feature supported by TGAC Browser with a novel interface. User Annotation: The TGAC Browser accepts user-defined annotations which can be persisted on the server and reloaded and shared at a later date. Off-the-shelf Installation: The prerequisites are Tomcat server and a standard Ensembl backend database to host sequence annotations and the TGAC Browser is ready to be used. Features Implementation • The Ensembl database schema underpins the TGAC Browser. • Ensembl database exposed to frontend with Java SQL data access objects. Data transfered to client via JSON and generates graphics via web browser JavaScript libraries. • We use property files so the TGAC Browser can be customised by editing a simple text file. • We use the jQuery, a JavaScript library and d3.js to generate interactive visualisations on the browser. • TGAC Browser can be customised to perform BLAST on local machine, NCBI or HPC cluster to make it faster for more users. 1. Database information: from meta_data 2. Easy Search: Search with reference, gene name, gene attributes, etc 3. BLAST Search: Integrates BLAST to search for sequence of interest 4. User Sessions: TGAC Browser supports user sessions 5. Assembly tracks 6. A standard gene track 7. Controls 8. Genomic Map with markers 2 3 4

An OLS regression model for context-aware tile prefetching in a web map cache

The increasing popularity of web map services has motivated the development of more scalable services in the spatial data infrastructures. Tiled map services have emerged as a scalable alternative to traditional map services. Instead of rendering map images on the fly, a collection of pre-generated image tiles can be served very fast from a server-side cache. However, during the start-up of the service, the cache is initially empty and users experience a poor quality of service. Tile prefetching attempts to improve hit rates by proactively fetching map images without waiting for client requests. While most popular prefetching policies in traditional web caching consider only the previous access history to make predictions, significant improvements could be achieved in web mapping by taking into account the background geographic information. This work proposes a regressive model to predict which areas are likely to be requested in the future based on spatial cross-correlation between an unconstrained catalog of geographic features and a record of past cache requests. Tiles that are anticipated to be most frequently requested can be pre-generated and cached for faster retrieval. Trace-driven simulations with several million cache requests from two different nation-wide public web map services in Spain demonstrate that accurate predictions and performance gains can be obtained with the proposed model.

A neural network based intelligent system for tile prefetching in web map services

Web mapping has become a popular way of distributing interactive digital maps over the internet. Instead of dynamically generating map images on the fly, those can be pre-generated and served from a server-side cache for faster retrieval. However, these caches can grow unmanageably in size when the cartography covers mid to large areas for multiple rendering scales. This forces modest organizations to use partial caches containing just a subset of the total tiles, and makes their services less attractive than other mapping services like Google Maps or Microsoft Bing Maps. This work proposes a neural-network-based intelligent system that predicts which areas are likely to be requested in the future from a catalog of geographic features and a short history of past requests. These priority regions can be used by a tile prefetching policy to achieve an optimal population of the cache. Neural networks are trained and validated using supervised learning with real data-sets from a public nation-wide web map service. Trace-driven simulations demonstrate that accurate long-term predictions, up to 90% in terms of cache-hit ratio, can be obtained with the proposed model by prefetching a low fraction, only the 20% of the total tiles, and with a short training period.

Performance Analysis and Improvement of Storage Virtualization in an OS-Online System

An OS-online system called TransCom is based on a virtual storage system that supports heterogeneous services of the operating system and applications online. In TransCom, OS and software which run in the client are stored on the centralized servers, while computing tasks are carried out by the clients, so the server is the bottleneck of the system performance. This paper firstly analyzes the characteristics of its real usage workload and builds a queuing model to locate the system bottlenecks. According to the study, the disk is the primary bottleneck, so an optimal two-level cache arrangement policy is developed on both the server and the client, which aims to avoid most of the server disk accesses. LRU algorithm is used in the client-side cache. A cache management algorithm called Frequency-based Multi-Priority Queues (FMPQ) proposed in this paper is used in the server-side cache. Experiment results show that the appropriate cache arrangement can significantly improve the capability of the TransCom server.

A Predictive Model for Frequently Viewed Tiles in a Web Map

Distributing pre-generated image tiles from a server-side cache is a fast and popular way of serving maps on the Web. However, these caches can grow unmanageably large for maps covering many scales over a broad extent. This research presents a model for determining high-priority geographic areas for tile caching. The model considers variables found in previous research to be of interest to Web map users, such as populated places, major roads, coastlines, and tourist attractions. The proportion of area, disk space, and time saved when creating a partial cache (consisting only of high-interest areas) and a cache of the entire map extent are compared to establish that marked savings are possible. Several applications of the model beyond tile caching are briefly discussed.

Towards building a conflict‐free mobile distributed file system

AbstractThe rising demand for mobile computing has created a need for an improved file system that supports mobile clients. Current file systems with support for mobility provide availability through file replicas that are cached at the client side. However, mobile clients may experience different obstacles with regards to the local cache, such as the limited network bandwidth, the intermittent connection, and serious conflicts when synchronizing back to the server. In this paper, we propose a novel mobile distributed file system design, which provides high availability and reliable storage for files and guarantees that file operations are executed regardless of concurrency and failure issues. The design is intended to fit mobile clients (e.g. PDAs and cell phones) that have limited storage space and cannot store all of the data they need, and yet require access to these data at all times. We adopt a server‐side caching in order to guarantee sufficient caching space to all mobile clients, and to ensure the availability of files in the case of clients' failures. We present our algorithm, describe its implementation, simulate its high availability functions, and report on its performance evaluation using a cluster of workstations. Our simulation results indicate clearly that our algorithm exhibits a significant degree of automation and conflict‐free mobile file system. Copyright © 2007 John Wiley & Sons, Ltd.

Improving the Performance of Online Auctions Through Server-side Activity-based Caching

Online auction sites have very specific workloads and user behavior characteristics. Previous studies on workload characterization conducted by the authors showed that (1) bidding activity on auctions increases considerably after 90% of an auction's life time has elapsed, (2) a very large percentage of auctions have a relatively low number of bids and bidders and a very small percentage of auctions have a high number of bids and bidders, (3) prices rise very fast after an auction has lasted more than 90% of its life time. Thus, if bidders are not able to successfully bid at the very last moments of an auction because of site overload, the final price may not be as high as it could be and sellers, and consequently the auction site, may lose revenue. In this paper, we propose server-side caching strategies in which cache placement and replacement policies are based on auction-related parameters such as number of bids placed or percent remaining time till closing time. A main-memory auction cache at the application server can be used to reduce accesses to the back-end database server. Trace-based simulations were used to evaluate these caching strategies in terms of cache hit ratio and cache efficiency. The performance characteristics of the best policies were then evaluated through experiments conducted on a benchmark online auction system.

Accelerating dynamic Web content generation

As a middle-tier, server-side caching engine, the dynamic content accelerator reduces dynamic page-generation processing delays by caching fragments of dynamically generated Web pages. This fragment-level solution, combined with intelligent cache management strategies, can significantly reduce the processing load on the Web application server, letting it handle higher user loads and thus significantly outperforming existing middle-tier caching solutions.