Prefetching Mechanism Research Articles

Optimizing Web Servers Using Page Rank Prefetching for Clustered Accesses

This paper presents a Page Rank based prefetching technique for accesses to Web page clusters. The approach uses the link structure of a requested page to determine the “most important” linked pages and to identify the page(s) to be prefetched. The underlying premise of our approach is that in the case of cluster accesses, the next pages requested by users of the Web server are typically based on the current and previous pages requested. Furthermore, if the requested pages have a lot of links to some “important” page, that page has a higher probability of being the next one requested. An experimental evaluation of the prefetching mechanism is presented using real server logs. The results show that the Page-Rank based scheme does better than random prefetching for clustered accesses, with hit rates of 90% in some cases.

Multipoint communications with speech mixing over IP network

Multipoint communications makes it possible to exchange information among a set of participants and also the key technique to realize a distributed multiparty collaboration communications system. In this paper, a multipoint communications strategy is discussed based on our H.323 multipoint conferencing system. A new architecture, Speech Scheduling and Mixing System (SSMS), is proposed for speech mixing, and its key techniques, such as the synchronous control of real time speeches, the pre-fetching mechanism of speech data, and the method of buffer flush are discussed in order to guarantee the continuity of the mixed speech in a limited complexity. Moreover, echo problem in the conferencing system is observed and a delayless subband AEC is used to cancel acoustic echo. Simulations show that our H.323 system sounds better in the multipoint conferencing mode than those video conferencing products available, such as Microsoft Netmeeting, which is based on the same H.323 specification.

Implementation and evaluation of a multifunctional telemedicine system in NTUH

In this article, we proposed a multifunctional telemedicine system supporting both telediagnosis and teleconsultation services. We attempted not only to insure that the implementation of this system satisfied most requirements, but also to evaluate the impact of the system. With regard to system architecture, we designed a unified multimedia database to store all types of data and used two kinds of network (ATM and ISDN) for different possible applications. As for data transmission, the REFRESH and PREFETCH mechanisms were implemented to enhance data transfer efficiency. A total of 1107 consultations employing the telemedicine system were performed during the past 3 years. This technology was used most frequently for radiology consultation (32.7%, n=362) and ultrasonic examination (19.5%, n=216). An evaluation of the impact on diagnosis (507 valid cases) indicated that the diagnosis in 80 cases (15.78%) were altered on the basis of second opinions from a medical center; and the number of patients transferred to the medical center was reduced from 24 (4.7%) to eight cases. Most of the rural-site physicians (97%) thought that they did benefit from specialists' experience and knowledge via the telemedicine system. Based on 431 valid questionnaires, the number of the patients with confidence in the telemedicine system at their local healthcare center increased from 72.6% to 87.5%. Overall, more than 90% of patients and physicians believed that the system was valuable and provided satisfactory services.

Data prefetch mechanisms

The expanding gap between microprocessor and DRAM performance has necessitated the use of increasingly aggressive techniques designed to reduce or hide the latency of main memory access. Although large cache hierarchies have proven to be effective in reducing this latency for the most frequently used data, it is still not uncommon for many programs to spend more than half their run times stalled on memory requests. Data prefetching has been proposed as a technique for hiding the access latency of data referencing patterns that defeat caching strategies. Rather than waiting for a cache miss to initiate a memory fetch, data prefetching anticipates such misses and issues a fetch to the memory system in advance of the actual memory reference. To be effective, prefetching must be implemented in such a way that prefetches are timely, useful, and introduce little overhead. Secondary effects such as cache pollution and increased memory bandwidth requirements must also be taken into consideration. Despite these obstacles, prefetching has the potential to significantly improve overall program execution time by overlapping computation with memory accesses. Prefetching strategies are diverse, and no single strategy has yet been proposed that provides optimal performance. The following survey examines several alternative approaches, and discusses the design tradeoffs involved when implementing a data prefetch strategy.

Reference implementation of scalable I/O low-level API on Intel Paragon

The Scalable I/O (SIO) Initiative’s Low-Level Application Programming Interface (SIO LLAPI) provides file system implementers with a simple low-Level interface to support high-level parallel I/O interfaces efficiently and effectively. This paper describes a reference implementation and the evaluation of the SIO LLAPI on the Intel Paragon multicomputer. The implementation provides the file system structure and striping algorithm, compatible with the Parallel File System (PFS) of Intel Paragon, and runs either inside the kernel or as a user level library. The scatter-gather addressing read/write, asynchronous I/O, client caching and prefetching mechanism, file access hint mechanism collective I/O and highly efficient file copy have been implemented. The preliminary experience shows that the SIO LLAPI provides opportunities of significant performance improvement and is easy to implement. Some high level file system interfaces and applications, such as PFS, ADIO and Hartree-Fock application, are also implemented on top of SIO. The performance of PFS is at least the same as that of Intel’s native PFS, and in many cases, such as small sequential file access, huge I/O requests and collective I/O, it is stable and much better. The SIO features help to support high level interfaces easily, quickly and more efficiently, and the cache, prefetching, hints are useful to get better performance based on different access models. The scalability and performance of SIO are limited by the network latency, network scalable bandwidth, memory copy bandwidth, memory size and pattern of I/O requests. The tradeoff between generality and efficiency should be considered in implementation.

NON-SEQUENTIAL INSTRUCTION CACHE PREFETCHING FOR MULTIPLE–ISSUE PROCESSORS

This paper presents a novel instruction cache prefetching mechanism for multiple-issue processors. Such processors at high clock rates often have to use a small instruction cache which can have significant miss rates. Prefetching from secondary cache or even memory can hide the instruction cache miss penalties, but only if initiated sufficiently far ahead of the current program counter. Existing instruction cache prefetching methods are strictly sequential and do not prefetch past conditional branches which may occur almost every clock cycle in wide-issue processors. In this study, multi-level branch prediction is used to overcome this limitation. By keeping branch history and target addresses, two methods are defined to predict a future PC several branches past the current branch. A prefetching architecture using such a mechanism is defined and evaluated with respect to its accuracy, the impact of the instruction prefetching on performance, and its interaction with sequential prefetching. Both PC-based and history-based predictors are used to perform a single-lookup prediction. Targeting an on-chip L2 cache with low latency, prediction for 3 branch levels is evaluated for a 4-issue processor and cache architecture patterned after the DEC Alpha-21164. It is shown that history-based predictor is more accurate, but both predictors are effective. The prefetching unit using them can be effective and succeeds when the sequential prefetcher fails. In addition, non-sequential prefetching is better at hiding latency due to earlier initiation. The two types of prefetching eliminate different types of misses and thus can be effectively combined to achieve better performance.

Guest Editors' Introduction Cache Memory And Related Problems: Enhancing And Exploiting The Locality

——————————F—————————— HE concept of cache memory has emerged as a solution for the ever increasing time domain gap between processor technology and memory technology. Since the very early works of Wilkes [13], the concept has evolved into a sophisticated system of hardware-implemented and software-implemented solutions. Actually, the best performance/complexity ratio is obtained through a synergistic interaction of hardware-based and software-based solutions. The efficiency of the caching system is achieved through appropriate exploitation of the principles of temporal and spatial locality. Traditionally, temporal locality means that the probability is relatively high that a data or an instruction item will be reused in the near future. Spatial locality means that the probability is relatively high that the next data or instruction item to be used is in some way neighboring the previously used data or instruction item. In traditional systems, temporal locality is exploited by keeping some of the most recently used data/instructions in the cache memory and by incorporating the cache hierarchy. Spatial locality is exploited by using larger cache blocks and by incorporating the prefetching mechanisms into the caching system. As technology gets more and more sophisticated, it has become obvious that a much better performance can be achieved through the incorporation of more sophisticated solutions for enhancing and exploiting of the locality present in the code or data. As microprocessors get more and more complex, cache design and performance become more and more impacted by the solutions utilized in other domains, like superpipelining, superscaling, multithreading, prediction, parallelization, etc. Implementation issues in modern microprocessor systems are getting new dimensions. The issues of most

Methods to improve performance of instruction prefetching through balanced improvement of two primary performance factors

Abstract The performance of conventional instruction prefetching mechanisms (IPMs) is analyzed in this paper based on two performance factors, i.e., the cache miss ratio and the average access time for successfully prefetched blocks. Although significant performance improvement (PI) can be obtained by improving these two factors, most conventional prefetching mechanisms improve only one factor out of these two factors. Fetching multiple blocks for a prefetch request and prefetching the sequentially next block together with the block that causes a cache miss in lookahead prefetching (LP) are proposed to improve both these factors. A new method to initiate a prefetch request earlier with no degradation of the prefetch accuracy is also presented for a memory system that is constructed as an interleaved memory. Performance evaluation is carried out through trace-driven simulation and the proposed prefetch scheme reduces 45–63% of the memory access delay time (MADT) for the cache system that does not perform any prefetching.

Non-referenced prefetch (NRP) cache for instruction prefetching

A new conceptual cache, NRP (non-referenced prefetch) cache, is proposed to improve the performance of instruction prefetch mechanisms which try to prefetch both the sequential and nonsequential blocks under the limited memory bandwidth. The NRP cache is used in storing prefetched blocks that were not referenced by the CPU, while these blocks were discarded in other previous prefetch mechanisms. By storing the non-referenced prefetch blocks in the NRP cache, both cache misses and memory traffic are reduced. A prefetch method to prefetch both the sequential and the nonsequential instruction paths is designed to utilise the effectiveness of the NRP cache. The results from trace-driven simulation show that this approach provides an improvement in memory access time than other prefetch methods. Particularly, the NRP cache is more effective in a lookahead prefetch mechanism that can hide longer memory latency. Also, the NRP cache reduces 50 – 112% of the additional memory traffic required to prefetch both instruction paths. This approach can achieve both the improved memory access time and the reduced memory traffic as a cost-effective cache design.

A software-controlled prefetching mechanism for software-managed TLBs

The TLB (Translation Lookaside Buffer) miss services have been concealed from operating systems, but some new RISC architectures manage the TLB in software. Since software-managed TLBs provide flexibility to an operating system in page translation, they are considered an important factor in the design of microprocessors for open system environments. However, software-managed TLBs suffer from larger miss penalty than hardware-managed TLBs, since they require more extra context switching overhead than hardware-managed TLBs. This paper introduces a new technique for reducing the miss penalty of software-managed TLBs by prefetching necessary TLB entries before being used. This technique is not inherently limited to specific applications. The key of this scheme is to perform the prefetch operations to update the TLB entries before first accesses so that TLB misses can be avoided. Using trace-driven simulation and a quantitative analysis, the proposed scheme is evaluated in terms of the miss rate and the total miss penalty. Our results show that the proposed scheme reduces the TLB miss rate by a factor of 6% to 77% due to TLB characteristics and page sizes. In addition, it is found that reducing the miss rate by the prefetching scheme reduces the total miss penalty and bus traffics in software-managed TLBs.

Digital archive system for radiologic images.

Use of a picture archiving and communication system (PACS) for clinical review requires rapid access to patient images. The authors have developed a second-generation archive system for a departmental PACS that optimizes access time for both current and historical images. The system is based on a composite staging mechanism that uses multiple storage media: magnetic disks, erasable magneto-optical disks, and write-once-read-many (WORM) disks. A two-tiered communication network composed of a standard Ethernet network and a gigabit bandwidth fiberoptic UltraNet network is used for distributing images to remote display stations. The system currently archives 1.5 Gbytes of radiologic images daily from three magnetic resonance imagers, two computed tomographic scanners, one computed radiographic system, and one laser film digitizer. The system features (a) multiple optical drives and robotic arms, which support concurrent archival and retrieval operations; (b) a stacking mechanism, which provides fast retrieval of current images; (c) a pre-fetch mechanism, which minimizes on-line image retrieval, hence relieving peak-hour workload of the optical disk library and the networks; (d) studies grouping and platter management, which optimize retrieval of a patient's images from multiple examinations; and (e) a job-prioritizing control, which minimizes waiting time for radiologists and referring physicians to review images at the display stations. This system is suitable for a large-scale PACS to serve the clinical and research staff.

Threaded prefetching: An adaptive instruction prefetch mechanism

We propose and analyze an adaptive instruction prefetch scheme, called threaded prefetching, that makes use of history information to guide the prefetching. The scheme is based on the observation that control flow paths are likely to repeat themselves. In the proposed scheme, we associate with each instruction block a number of threads that indicate the instruction blocks that have been brought into the cache by the current block. These threads later trigger the prefetching of the indicated instruction blocks once the instruction block containing them are re-accessed by the processor. These pointers, in effect, encode the causal relationship between an instruction block and the instruction blocks that have been brought into the cache by the block. The results from trace-driven simulations using SPEC benchmarks show that the proposed scheme improves the prefetch accuracy by more than 100% on average for 32 Kb cache. They also show that the scheme significantly (80% on average for 32 Kb cache) improves the CPI (Clocks Per Instruction) due to instruction references over the sequential prefetching.

Multiple prefetch adaptive disk caching

A new disk caching algorithm is presented that uses an adaptive prefetching scheme to reduce the average service time for disk references. Unlike schemes which simply prefetch the next sector or group of sectors, this method maintains information about the order of past disk accesses which is used to accurately predict future access sequences. The range of parameters of this scheme is explored, and its performance is evaluated through trace-driven simulation, using traces obtained from three different UNIX minicomputers. Unlike disk trace data previously described in the literature, the traces used include time stamps for each reference. With this timing information-essential for evaluating any prefetching scheme-it is shown that a cache with the adaptive prefetching mechanism can reduce the average time to service a disk request by a factor of up to three, relative to an identical disk cache without prefetching.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Using horizontal prefetching to circumvent the jump problem

The principle of a novel prefetching mechanism, horizontal demand prefetching, is presented. The mechanism allows deep prefetching without jump related misses by prefetching horizontally across independent instruction streams. The scheme can achieve high memory utilization at the expense of processor utilization. The mechanism permits very rapid context switching with no overhead for a hardware-limited number of tasks. The design and performance of a 16-b prototype machine is presented. The prototype exhibits a nonlinear relationship between the number of running streams and processor performance. This saturating performance-tasks relationship suggests that operations like process synchronization and interprocessor communication could be implemented very efficiently. Stalled streams need not greatly affect processor throughput. >

The case for the sustained performance computer architecture

The thesis of this report is that, in addition to existing methods for improving computer performance, high performance systems of the 90:0090's and beyond will benefit from a sustained performance architecture (SPA). This architecture, to be introduced informally here, is independent of technology and instruction set, and can be incorporated in machines without sacrificing binary comparitiblity. The purpose of SPA is to keep the instruction pipeline of a uniprocessor full to the greatest possible extend; this efficiency comes at the cost of increased hardware complexity in the prefetch mechanism and software complexity in the use of program flow information.

The instruction parsing microarchitecture of CVAX microprocessor

CVAX is a single chip, CMOS VLSI VAX microprocessor. Several microarchitectural innovations helped achieve the desired performance goal of this machine. In particular, the instruction parsing and prefetching mechanism is different from other VAX implementations. This new instruction parsing microarchitecture is discussed in this paper.

A compiler-driven supercomputer

The overall performance of supercomputers is slow compared to the speed of their underlying logic technology. This discrepancy is due to several bottlenecks: memories are slower than the CPU, conditional jumps limit the usefulness of pipelining and pre-fetching mechanisms, and functional-unit parallelism is limited by the speed of hardware scheduling. This paper describes a supercomputer architecture called Ring of Pre-fetch Elements (ROPE) that attempts to solve the problems of memory latency and conditional jumps, without hardware scheduling. ROPE consists of a very pipelined CPU data path with a new instruction pre-fetching mechanism that supports general multi-way conditional jumps. An optimizing compiler based on a global code transformation technique (Percolation Scheduling or PS) gives high performance without scheduling hardware.