Instruction Stream Research Articles

Modified Architectural Support for Predicate Execution of Instruction level Parallelism

Utilizing speculative execution alone to extract instruction level parallelism in the presence of branches has performance limitation .The fundamental limitation is that speculation eliminates dependencies between instructions and branches, but does not remove the branches themselves. To overcome this drawback, predicate execution is investigated. Predicate or guarded execution enables a compiler to eliminate branches from the instruction stream. As a result, many of the difficulties introduced by branches can be eliminated. This paper addresses the architectural support required to accomplish predicate execution. The architectural extensions required to provide efficient support for the predicated execution are discussed.

Performance Optimization of Hyperblock Using Predicate Compilation Technique

Predicate execution provides a large number of opportunities to enhance and expose ILP in the presence of branches. However, as with speculative execution, an aggressive compiler is required to realize most of the performance advantages. Compiler Optimization and transformation techniques focus on eliminating branches from the instruction stream and overlapping the execution of multiple control flow paths using the conditional execution capabilities provided by predication. The compiler support for predicated execution is based on a new structure referred to as the hyperblock. . Hyperblocks are a generalized form of Superblock that takes advantage of both predicated and speculative execution. This paper discusses the hyperblock compilation techniques. The formation procedure of hyperblock is described first. Secondly, the extensions to traditional optimization, instruction scheduling, and register allocation techniques to enable them to work on hyperblocks. The presence of predicates introduces new challenges into the compiler backend to understand the meaning of predicates, take the advantage of the relations among predicates, and perform transformations in the presence of predicates. Finally, a set of four new optimizations designed specially for improving the performance of predicated code.

A Novel instruction stream buffer for VLIW architectures

The instruction compression mechanism used to solve the drawbacks of traditional very long instruction word (VLIW) architectures often leads to poor code density in the instruction cache, which causes the irregular lengths of long instructions to cross the different cache line. These split long instructions cannot be fetched simultaneously, which creates a bottleneck for VLIW architectures. This paper proposes a buffing mechanism which can slide the split long instruction as a continuous form to offer better efficiency in instruction fetching. This approach helps maintain the behaviors of the software pipeline technology, which schedules iterative instructions to enhance the performance of streaming processing for VLIW architectures. In the proposed mechanism, the instruction stream buffer stores the repeat block completely and suspends as far as possible the cache access to reduce access time. The advantages of repeatedly issuing instructions in the instruction buffer and preventing split long instructions, can substantially improve the performance in fetching instructions. Simulation results show that the mechanism is efficient at the instruction level for the basic DSP/IMG library by improving performance by 35% on average.

A parallel algorithm for generating bicompatible elimination orderings of proper interval graphs

In this paper, we first show how a certain ordering of vertices, called bicompatible elimination ordering (BCO), of a proper interval graph can be used to solve efficiently several problems in proper interval graphs. We, then, propose an NC parallel algorithm (i.e., polylogarithmic-time employing a polynomial number of processors) in SIMD CRCW PRAM (Single Instruction Stream Multiple Data Stream Concurrent Read Concurrent Write Parallel Random Access Machine) model of parallel computation to compute a BCO of a proper interval graph. To the best of our knowledge, this is the first NC parallel algorithm to compute a BCO of a proper interval graph.

How to simulate 1000 cores

This paper proposes a novel methodology to efficiently simulate shared-memory multiprocessors composed of hundreds of cores. The basic idea is to use thread-level parallelism in the software system and translate it into corelevel parallelism in the simulated world. To achieve this, we first augment an existing full-system simulator to identify and separate the instruction streams belonging to the different software threads. Then, the simulator dynamically maps each instruction flow to the corresponding core of the target multi-core architecture, taking into account the inherent thread synchronization of the running applications. Our simulator allows a user to execute any multithreaded application in a conventional full-system simulator and evaluate the performance of the application on a many-core hardware. We carried out extensive simulations on the SPLASH-2 benchmark suite and demonstrated the scalability up to 1024 cores with limited simulation speed degradation vs. the single-core case on a fixed workload. The results also show that the proposed technique captures the intrinsic behavior of the SPLASH-2 suite, even when we scale up the number of shared-memory cores beyond the thousand-core limit.

Low Power Reconfiguration Technique for Coarse-Grained Reconfigurable Architecture

Coarse-grained reconfigurable architectures (CGRAs) require many processing elements (PEs) and a configuration memory unit (configuration cache) for reconfiguration of its PE array. Although this structure is meant for high performance and flexibility, it consumes significant power. Specially, power consumption by configuration cache is explicit overhead compared to other types of intellectual property (IP) cores. Reducing power is very crucial for CGRA to be more competitive and reliable processing core in embedded systems. In this paper, we propose a reusable context pipelining (RCP) architecture to reduce power-overhead caused by reconfiguration. It shows that the power reduction can be achieved by using the characteristics of loop pipelining, which is a multiple instruction stream, multiple data stream (MIMD)-style execution model. RCP efficiently reduces power consumption in configuration cache without performance degradation. Experimental results show that the proposed approach saves much power even with reduced configuration cache size. Power reduction ratio in the configuration cache and the entire architecture are up to 86.33% and 37.19%, respectively, compared to the base architecture.

Rock: A High-Performance Sparc CMT Processor

Rock, Sun's third-generation chip-multithreading processor, contains 16 high-performance cores, each of which can support two software threads. Rock uses a novel checkpoint-based architecture to support automatic hardware scouting under a load miss, speculative out-of-order retirement of instructions, and aggressive dynamic hardware parallelization of a sequential instruction stream. It is also the first processor to support transactional memory in hardware.

Phantom-BTB

Modern processors use branch target buffers (BTBs) to predict the target address of branches such that they can fetch ahead in the instruction stream increasing concurrency and performance. Ideally, BTBs would be sufficiently large to capture the entire working set of the application and sufficiently small for fast access and practical on-chip dedicated storage. Depending on the application, these requirements are at odds. This work introduces a BTB design that accommodates large instruction footprints without dedicating expensive onchip resources. In the proposed Phantom-BTB (PBTB) design, a conventional BTB is augmented with a virtual table that collects branch target information as the application runs. The virtual table does not have fixed dedicated storage. Instead, it is transparently allocated, on demand, in the on-chip caches, at cache line granularity. The entries in the virtual table are proactively prefetched and installed in the dedicated conventional BTB, thus, increasing its perceived capacity. Experimental results with commercial workloads under full-system simulation demonstrate that PBTB improves IPC performance over a 1K-entry BTB by 6.9% on average and up to 12.7%, with a storage overhead of only 8%. Overall, the virtualized design performs within 1% of a conventional 4K-entry, single-cycle access BTB, while the dedicated storage is 3.6 times smaller.

Phantom-BTB

Modern processors use branch target buffers (BTBs) to predict the target address of branches such that they can fetch ahead in the instruction stream increasing concurrency and performance. Ideally, BTBs would be sufficiently large to capture the entire working set of the application and sufficiently small for fast access and practical on-chip dedicated storage. Depending on the application, these requirements are at odds. This work introduces a BTB design that accommodates large instruction footprints without dedicating expensive onchip resources. In the proposed Phantom-BTB (PBTB) design, a conventional BTB is augmented with a virtual table that collects branch target information as the application runs. The virtual table does not have fixed dedicated storage. Instead, it is transparently allocated, on demand, in the on-chip caches, at cache line granularity. The entries in the virtual table are proactively prefetched and installed in the dedicated conventional BTB, thus, increasing its perceived capacity. Experimental results with commercial workloads under full-system simulation demonstrate that PBTB improves IPC performance over a 1K-entry BTB by 6.9% on average and up to 12.7%, with a storage overhead of only 8%. Overall, the virtualized design performs within 1% of a conventional 4K-entry, single-cycle access BTB, while the dedicated storage is 3.6 times smaller.

A tightly coupled finite field arithmetic hardware in an FPGA-based embedded processor core for elliptic curve cryptography

This work presents the implementation of a tightly-coupled hardware architectural enhancement to the Altera FPGA-based Nios II embedded processor. The goal is to accelerate finite field arithmetic operations in the binary fields of F 2 163 and F 2 193, for application in a high-performance embedded system implementing elliptic curve cryptography (ECC). The concept is to augment the embedded processor with a few custom instructions for fast finite field arithmetic operations. Instead of a coprocessor, hardware acceleration of the arithmetic operation is provided by custom logic tightly coupled to the processor core and directly controlled by the instruction stream. This concept, which may be considered as a hardware-software co-design approach, is evaluated by prototyping an elliptic curve cryptosystem (ECC) on an Altera Stratix FPGA development board. Experimental results shows that for the point multiplication operation, which is the core operation in an ECC computation, the implementation with custom instructions and tightly-coupled hardware is about 50% faster than the coprocessor-based hardware.

VLIW processors: once blue sky, now commonplace

Very long instruction word (VLIW) is an architectural style that one of the authors-Josh Fisher-proposed about 30 years ago to speed up computers and otherwise enhance their performance. Those listening to Fisher's first public "blue-sky" expositions of this technology in the early 1980s did not generally expect it to succeed. Indeed, they would have been stunned to hear of the success these processors are enjoying today, especially as embedded processors, designed to perform special-purpose functions, usually in real time, in some kind of hardware. VLIWs obtain high performance from ordinary programs in a way that is simple to describe: Instead of issuing a single operation from an instruction stream in a given cycle, a VLIW processor issues many of them together in a single execution stream, and it issues new operations before old ones have finished.

A Taxonomy of Reconfigurable Single‐/Multiprocessor Systems‐on‐Chip

Runtime adaptivity of hardware in processor architectures is a novel trend, which is under investigation in a variety of research labs all over the world. The runtime exchange of modules, implemented on a reconfigurable hardware, affects the instruction flow (e.g., in reconfigurable instruction set processors) or the data flow, which has a strong impact on the performance of an application. Furthermore, the choice of a certain processor architecture related to the class of target applications is a crucial point in application development. A simple example is the domain of high‐performance computing applications found in meteorology or high‐energy physics, where vector processors are the optimal choice. A classification scheme for computer systems was provided in 1966 by Flynn where single/multiple data and instruction streams were combined to four types of architectures. This classification is now used as a foundation for an extended classification scheme including runtime adaptivity as further degree of freedom for processor architecture design. The developed scheme is validated by a multiprocessor system implemented on reconfigurable hardware as well as by a classification of existing static and reconfigurable processor systems.

Very long instruction work architectures and the ELI-512

By compiling ordinary scientific applications programs with a radical technique called trace scheduling, we are generating code for a parallel machine that will run these programs faster than an equivalent sequential machine - we expect 10 to 30 times faster. Trace scheduling generates code for machines called very long instruction work architectures. In very long instruction word machines, many statically scheduled, tightly coupled, fine-grained operations exwcute in parallel with a single instruction stream. VLIWs are more parallel extensions of several current architectures. These current architectures have never cracked a fundamental barrier. The speedup they get from parallelism is never more than a factor of 2 to 3. Not htat we couldn't build more parallel machines of this type: but until trace scheduling we did't know how to generate code for them. Trace scheduling finds sufficient paralleslism in ordinary code to justify thinking about a highly parallel VLIW. At Yale we are actualy buildingone. Our machine, the ELI-512 has a horizontal instruction word of over 500 bits and will do 10 to 30 RISC-level operations per cyple [Patterson 82]. ELI stands for Enormously Longword Instructions; 512 is the size of the instruction word we hope to achieve. (The current design has a 1200-bit instruction word).

Uniting extrinsic vectorization and shell structure for efficient SIMD evaluation of electron repulsion integrals

Future computer architectures are likely to feature greater reliance on single instruction multiple data (SIMD) processing for high throughput processing of data-intensive workloads. For algorithms that rely heavily on electron repulsion integrals (ERIs), exploitation of SIMD processing requires extrinsic vectorization, i.e. the sorting of ERIs into sets with equivalent class that may be computed with an identical instruction stream. Such sorting is incongruous with the commonly exploited shell structure whereby ERI are generated over shells such that initialization/bootstrap values may be reused, yielding significant savings in ERI evaluation time. In this work, we discuss how extrinsic vectorization may be unified with shell structure through the exploitation of memory access locality.

Dynamic Instruction Scheduling in a Trace-based Multi-threaded Architecture

Simulation results are presented using the hardware-implemented, trace-based dynamic instruction scheduler of our single process DTSVLIW architecture to schedule instructions from several processes into multiple streams of VLIW instructions for execution by a wide-issue, simultaneous multi-threading (SMT) execution engine. The scheduling process involves single instruction execution of each process, dynamically scheduling executed instructions into blocks of VLIW instructions cached for subsequent SMT execution: SMT provides a mechanism to reduce the impact of horizontal and vertical waste, and variable memory latencies, seen in the DTSVLIW. Preliminary experiments explore this extended model. Results achieve PE utilization of up to 87% on a 4-thread, 1-scalar, 8 PE design, with speed-ups of up to 6.3 that of a single processor. Noticeably it only needs a single scalar process to be scheduled at any time, with main memory fetches being 1-4% that of a single processor.

A low-cost recovery scheme for dynamically scheduled processors

Although today's branch predictors show high accuracy, the branch misprediction penalty is getting larger due to aggressive speculation and deeper pipelining. In order to reduce the miss penalty, we propose a fast and low-cost branch recovery scheme using the incremental register renaming (IRR) and the bit-vector based rename map table (BVMT). The IRR enforces the destination register number of the instruction stream to appear in non-decreasing order. With this incremental property of the IRR, the BVMT recovery scheme completely eliminates the roll-back overhead on branch misprediction. Thus, the instruction fetcher does not stop and it fetches instructions from the correct path immediately after the misprediction detected. The goal of our scheme is to prevent a processor from flushing the pipeline, even under branch misprediction. Consequently, the BVMT instantly reconstructs the map table to any mispredicted branch and it outperforms the conventional approach by an average of 10.93%.

What is multithreading and multi-core?

Is multithreading better than multi-core? Is multi-core better than multithreading? The fact is that the best vehicle for a given application might have one, the other or both. Or neither. They are independent (but complementary) design decisions. As multithreaded processors and multi-core chips become the norm, architects and designers of digital systems need to understand their respective attributes, advantages and disadvantages. Both multithreading and multi-core approaches exploit the concurrency in a computational workload. The cost, in silicon, energy and complexity, of making a CPU run a single instruction stream ever faster goes up nonlinearly and eventually hits a wall imposed by the physical limitations of circuit technology. That wall keeps moving out a little farther every year, but cost- and power-sensitive designs are constrained to follow the bleeding edge from a safe distance. Fortunately, virtually all computer applications have some degree of concurrency: At least some of the time, two or more independent tasks need to be performed simultaneously. Taking advantage of concurrency to improve computing performance and efficiency isn't always trivial, but it's certainly easier than violating the laws of physics.

What is multithreading and multi-core?

Is multithreading better than multi-core? Is multi-core better than multithreading? The fact is that the best vehicle for a given application might have one, the other or both. Or neither. They are independent (but complementary) design decisions. As multithreaded processors and multi-core chips become the norm, architects and designers of digital systems need to understand their respective attributes, advantages and disadvantages. Both multithreading and multi-core approaches exploit the concurrency in a computational workload. The cost, in silicon, energy and complexity, of making a CPU run a single instruction stream ever faster goes up nonlinearly and eventually hits a wall imposed by the physical limitations of circuit technology. That wall keeps moving out a little farther every year, but cost- and power-sensitive designs are constrained to follow the bleeding edge from a safe distance. Fortunately, virtually all computer applications have some degree of concurrency: At least some of the time, two or more independent tasks need to be performed simultaneously. Taking advantage of concurrency to improve computing performance and efficiency isn't always trivial, but it's certainly easier than violating the laws of physics.

Exploring the performance limits of simultaneous multithreading for memory intensive applications

Simultaneous multithreading (SMT) has been proposed to improve system throughput by overlapping instructions from multiple threads on a single wide-issue processor. Recent studies have demonstrated that diversity of simultaneously executed applications can bring up significant performance gains due to SMT. However, the speedup of a single application that is parallelized into multiple threads, is often sensitive to its inherent instruction level parallelism (ILP), as well as the efficiency of synchronization and communication mechanisms between its separate, but possibly dependent threads. Moreover, as these separate threads tend to put pressure on the same architectural resources, no significant speedup can be observed. In this paper, we evaluate and contrast thread-level parallelism (TLP) and speculative precomputation (SPR) techniques for a series of memory intensive codes executed on a specific SMT processor implementation. We explore the performance limits by evaluating the tradeoffs between ILP and TLP for various kinds of instruction streams. By obtaining knowledge on how such streams interact when executed simultaneously on the processor, and quantifying their presence within each application's threads, we try to interpret the observed performance for each application when parallelized according to the aforementioned techniques. In order to amplify this evaluation process, we also present results gathered from the performance monitoring hardware of the processor.

Enlarging Instruction Streams

The stream fetch engine is a high-performance fetch architecture based on the concept of instruction stream. We call stream to a sequence of instructions from the target of a taken branch to the next taken branch, potentially containing multiple basic blocks. The long size of instruction streams makes it possible for the stream fetch engine to provide high fetch bandwidth and to hide the branch predictor access latency, leading to performance results close to a trace cache at lower implementation cost and complexity. Therefore, enlarging instruction streams is an excellent way for improving the stream fetch engine. In this paper, we present several hardware and software mechanisms focused on enlarging those streams that finalize at particular branch types. However, our results point out that focusing on particular branch types is not a good strategy due to Amdahl's law. Consequently, we propose the multiple stream predictor, a novel mechanism that deals with all branch types by combining single streams into long virtual streams. This proposal tolerates the prediction table access latency without requiring the complexity caused by additional hardware mechanisms like prediction overriding. Moreover, it provides high performance results, which are comparable to state-of-the-art fetch architectures, but with a simpler design that consumes less energy.