On-chip Instruction Cache Research Articles

On the characterization and optimization of system-level vulnerability for instruction caches in embedded processors

With continuous scaling down of the semiconductor technology, the soft errors induced by energetic particles have become an increasing challenge in designing current and next-generation reliable microprocessors. Due to their large share of the transistor budget and die area, cache memories suffer from an increasing vulnerability against soft errors. Previous work based on the vulnerability factor (VF) analysis proposed analytical models to evaluate the reliability of on-chip data and instruction caches. However, we have no possession of a system-level study on the vulnerability of instruction caches. In this paper, we propose a new analytical model to estimate the system-level vulnerability factor for on-chip instruction caches in embedded processors. In our model, the error masking/detection effects in instructions based on the Instruction Set Architecture (ISA) are studied. Our experimental results show that the self-error-masking/detection in instructions will reduce the VF of the instruction caches compared to the previous study. We also exemplify our design methodology by proposing several optimizing schemes to improve the reliability. Benchmarking is carried out to demonstrate the effectiveness of our vulnerability model and optimization approach, which can provide an insightful guidance for the future reliable instruction cache and ISA design.

Reducing Power and Energy Consumption of Nonvolatile Microcontrollers with Transparent On-Chip Instruction Cache

Demands for low-energy microcontrollers have been increasing in recent years. Since most microcontrollers achieve user programmability by integrating nonvolatile (NV) memories such as flash memories for storing their programs, the large power consumption required in accessing an NV memory has become a major problem. This problem becomes critical when the power supply voltage of NV microcontrollers is decreased. We can solve this problem by introducing an instruction cache, thus reducing the access frequency of the NV memory. Unlike general-purpose microprocessors, microcontrollers used for real-time applications in embedded systems must accurately calculate program execution time prior to its execution. Therefore, we introduce a “transparent” instruction cache, which does not change the existing NV microcontroller’s cycle-level execution time, for reducing power and energy consumption, but not for improving the processing speed. We have conducted detailed microar chitecture design based on the architecture of a major industrial microcontroller, and we evaluated power and energy consumption for several benchmark programs. Our evaluation shows that the proposed instruction cache can successfully reduce energy consumption in a fairly wide range of practical NV microcontroller configurations.

DLIC

With the explosive proliferation of embedded systems, especially through countless portable devices and wireless equipment used, embedded systems have become indispensable to the modern society and people's life. Those devices are often battery driven. Therefore, low energy consumption in embedded processors is important and becomes critical in step with the system complexity. The on-chip instruction cache (I-cache) is usually the most energy-consuming component on the processor chip due to its large size and frequent access operations. To reduce such energy consumption, the existing loop cache approaches use a tiny decoded cache to filter the I-cache access and instruction decode activity for repeated loop iterations. However, such designs are effective for small and simple loops, and only suitable for DSP kernel-like applications. They are not effectual for many embedded applications where complex loops are common. In this article, we propose a decoded loop instruction cache (DLIC) that is small, hence energy efficient, yet can capture most loops, including large nested ones with branch executions, so that a significant amount of I-cache accesses and instruction decoding can be eradicated. The experiments on a set of embedded benchmarks show that our proposed DLIC scheme can reduce energy consumption by up to 87% as compared to normal cache-only design. On average, 66% energy can be saved on instruction fetching and decoding, while at a performance overhead of only 1.4%.

DLIC

With the explosive proliferation of embedded systems, especially through countless portable devices and wireless equipment used, embedded systems have become indispensable to the modern society and people's life. Those devices are often battery driven. Therefore, low energy consumption in embedded processors is important and becomes critical in step with the system complexity. The on-chip instruction cache (I-cache) is usually the most energy-consuming component on the processor chip due to its large size and frequent access operations. To reduce such energy consumption, the existing loop cache approaches use a tiny decoded cache to filter the I-cache access and instruction decode activity for repeated loop iterations. However, such designs are effective for small and simple loops, and only suitable for DSP kernel-like applications. They are not effectual for many embedded applications where complex loops are common. In this article, we propose a decoded loop instruction cache (DLIC) that is small, hence energy efficient, yet can capture most loops, including large nested ones with branch executions, so that a significant amount of I-cache accesses and instruction decoding can be eradicated. The experiments on a set of embedded benchmarks show that our proposed DLIC scheme can reduce energy consumption by up to 87% as compared to normal cache-only design. On average, 66% energy can be saved on instruction fetching and decoding, while at a performance overhead of only 1.4%.

A Trace-Capable Instruction Cache for Cost-Efficient Real-Time Program Trace Compression in SoC

This paper presents a novel approach to make the on-chip instruction cache of a SoC to function simultaneously as a regular instruction cache and a real-time program trace compressor, named trace-capable cache (TC-cache). It is accomplished by exploiting the dictionary feature of the instruction cache with a small support circuit attached to the side of the cache. Compared with related work, this work has the advantage of utilizing the existing instruction cache, which is indispensable in modern SoCs, and thus saves significant amount of hardware resource and power consumption. The TC-cache can be configured to work simultaneously as the instruction cache and the trace compressor, named the online mode, or exclusively as the trace compressor, named the bypass mode. The RTL implementation of a 4 KB trace-capable instruction cache, a 4 KB data cache, and an academic ARM processor core has been accomplished. The experiments show that the TC-cache achieves average compression ratio of 90 percent with a very small hardware overhead of 3,652 gates (1.1 percent). It takes only 0.2 percent additional system power for the online mode operation. In addition, the trace support circuit does not impair the global critical path. Therefore, the proposed approach is a highly feasible on-chip debugging/monitoring solution for SoCs, even for cost-sensitive ones such as consumer electronics. Furthermore, the same concept can be applied to the data cache to compress the data address trace as well.

Dynamically Adaptive I-Cache Partitioning for Energy-Efficient Embedded Multitasking

The ever increasing importance of battery-powered devices coupled with high performance requirements and shrinking process geometries have further exacerbated the problem of energy efficiency in modern embedded systems. The cache memories are a major contributor to the system power consumption, and as such have been a primary target for energy reduction techniques. Recent advances in configurable cache architectures have enabled an entirely new set of approaches for application-driven energy- and cost-efficient cache resource utilization. We propose a run-time and adaptive instruction cache partitioning methodology, which leverages configurable cache architectures to achieve an energy- and performance-conscious adaptive mapping of instruction cache resources to tasks in dynamic multi-task workloads sharing a processor core trough preemptive multitasking. Sizable leakage and dynamic power reductions are achieved with only a negligible and system-controlled performance impact. The methodology assumes no prior information regarding the dynamics and the structure of the workload. As the proposed dynamic cache partitioning alleviates the adverse effects of cache interference, performance is maintained very close to the baseline case, while achieving 50%-80% reductions in dynamic and leakage power for the on-chip instruction cache memory.

An on-chip instruction cache design with one-bit tag for low-power embedded systems

On-chip instruction cache is a potential power hungry component in embedded systems due to its large chip area and high access-frequency. Aiming at reducing power consumption of the on-chip cache, we propose a Reduced One-Bit Tag Instruction Cache ( ROBTIC), where the cache size is judiciously reduced and the cache tag field only contains the least significant bit of the full-tag. We develop a cache operational control scheme for ROBTIC so that with the one-bit cache tag, the program locality can still be efficiently exploited. For applications where most of the memory accesses are localized, our cache can achieve similar performance as a traditional full-tag cache; however, the power consumption of the cache can be significantly reduced due to the much smaller cache size, narrower tag array (just one bit), and tinier tag comparison circuit being used. Experiments on a set of benchmarks implemented in CMOS 180 nm process technology demonstrate that our proposed design can reduce up to 27.3% dynamic power consumption and 30.9% area of the traditional cache when the cache size is fixed at 32 instructions, which outperforms the existing partial-tag based cache design. With the cache size customization, a further 47.8% power saving can be achieved. Our experimental results also show that when implemented in the deep sub-micron technologies where the leakage power is not ignorable, our design is still efficient – a coherent power saving trend (about 22%) has been observed for technologies from 130 nm down to 65 nm.

Dynamic scratchpad memory management for code in portable systems with an MMU

In this work, we present a dynamic memory allocation technique for a novel, horizontally partitioned memory subsystem targeting contemporary embedded processors with a memory management unit (MMU). We propose to replace the on-chip instruction cache with a scratchpad memory (SPM) and a small minicache. Serializing the address translation with the actual memory access enables the memory system to access either only the SPM or the minicache. Independent of the SPM size and based solely on profiling information, a postpass optimizer classifies the code of an application binary into a pageable and a cacheable code region. The latter is placed at a fixed location in the external memory and cached by the minicache. The former, the pageable code region, is copied on demand to the SPM before execution. Both the pageable code region and the SPM are logically divided into pages the size of an MMU memory page. Using the MMU's pagefault exception mechanism, a runtime scratchpad memory manager (SPMM) tracks page accesses and copies frequently executed code pages to the SPM before they get executed. In order to minimize the number of page transfers from the external memory to the SPM, good code placement techniques become more important with increasing sizes of the MMU pages. We discuss code-grouping techniques and provide an analysis of the effect of the MMU's page size on execution time, energy consumption, and external memory accesses. We show that by using the data cache as a victim buffer for the SPM, significant energy savings are possible. We evaluate our SPM allocation strategy with fifteen applications, including H.264, MP3, MPEG-4, and PGP. The proposed memory system requires 8% less die are compared to a fully-cached configuration. On average, we achieve a 31% improvement in runtime performance and a 35% reduction in energy consumption with an MMU page size of 256 bytes.

Compact and efficient code generation through program restructuring on limited memory embedded DSPs

Many embedded systems such as digital cameras, digital radios, high-resolution printers, cellular phones, etc., involve a heavy use of signal processing and are thus based on digital signal processors (DSPs). DSPs such as the TMS320C2x and the DSP5600x have irregular data paths that typically result due to application specific needs (such as chaining multiply-accumulate operations, etc.). Efficient code generation for such embedded DSP processors is a challenging problem. The stringent requirements such as tight memory constraints and fast response time result in the need for a compact and efficient code. In this paper, we address the problem of generating a compact and efficient code for embedded DSP processors. Most of the DSP instruction set architectures (ISAs) feature intrainstruction parallelism (IIP), enabling individual operations to be executed in parallel by generating a complex instruction. A reduction in generated code size and improved performance can be achieved by exploiting this parallelism present in such ISAs. In this paper, we present a code restructuring technique to fully exploit this parallelism through maximal utilization of the complex instructions present in the instruction set. We formulate this as a maximal benefit code restructuring problem, which is to derive the arrangement of statements to maximally exploit IIP without violating data dependencies. This problem is equivalent to the precedence constrained Hamiltonian path problem for directed acyclic graphs and the traveling salesman problem in general, both of which are NP-hard. In this paper, we present an optimal algorithm to solve the problem. We have implemented this optimal algorithm in a compiler targeted to generate code for the TMS320C25 DSP. We tested our framework on a number of benchmarks and found that the performance of the generated code (measured in dynamic instruction cycle counts) improves by as much as 9.9% with an average of 4%. The average code-size reduction over code compiled without exploiting parallelism is 2.9%. We also studied the effect of loop unrolling on the available IIP. An on-chip instruction cache can be effectively utilized by unrolling loops such that generated code fully occupies the memory. The benefit is reduction in dynamic instruction count due to the higher number of complex instructions generated. We found that by unrolling loop by four times to fit available on-chip instruction cache, the dynamic instruction counts reduce by as much as 9.9 %.

Instruction cache performance of a commercial workload on the Motorola 88110 microprocessor

Commercial workloads, such as transaction processing, stress the memory hierarchies of current computers. TPC-B is a transaction processing benchmark that generates significant operating system and database activity, and a TPC-B instruction trace is analyzed for the Motorola 88110 microprocessor. Instruction set usage reveals one branch in every four to five instructions, and instruction cache performance indicates that this branching behavior causes a large number of words to be brought in by cache refills but never used. Code reorganization optimizations to reduce unused words and avoid cache misses are simulated and found to reduce misses in the on-chip instruction cache by almost one fifth.

Instruction prefetching of systems codes with layout optimized for reduced cache misses

High-performing on-chip instruction caches are crucial to keep fast processors busy. Unfortunately, while on-chip caches are usually successful at intercepting instruction fetches in loop-intensive engineering codes, they are less able to do so in large systems codes. To improve the performance of the latter codes, the compiler can be used to lay out the code in memory for reduced cache conflicts. Interestingly, such an operation leaves the code in a state that can be exploited by a new type of instruction prefetching: guarded sequential prefetching.The idea is that the compiler leaves hints in the code as to how the code was laid out. Then, at run time, the prefetching hardware detects these hints and uses them to prefetch more effectively. This scheme can be implemented very cheaply: one bit encoded in control transfer instructions and a prefetch module that requires minor extensions to existing next-line sequential prefetchers. Furthermore, the scheme can be turned off and on at run time with the toggling of a bit in the TLB. The scheme is evaluated with simulations using complete traces from a 4-processor machine. Overall, for 16-Kbyte primary instruction caches, guarded sequential prefetching removes, on average, 66% of the instruction misses remaining in an operating system with an optimized layout, speeding up the operating system by 10%. Moreover, the scheme is more cost-effective and robust than existing sequential prefetching techniques.

An array processor for general purpose digital image compression

A new VLSI processor (DIP chip) for image compression is presented which combines principles of multipipeline and array processing. The device is not specific to any one image compression algorithm and can be regarded as a general purpose processor. The chip has been implemented using a CMOS 1.0-/spl mu/m process on a 14.4/spl times/13.5-mm/sup 2/ die. An internal clock frequency of 40 MHz results in 1.2/spl times/10/sup 9/ operations/s on 8-bit data. Solutions to problems associated with the large bandwidth required, for both image data and instruction streams, is the main aim of the paper. The necessary problem of increasing the array clock frequency relative to the input/output clock frequency without the need for a large on-chip instruction cache or fast external clock speeds is also addressed. >

Optimal allocation of on-chip memory for multiple-API operating systems

The allocation of die area to different processor components is a central issue in the design of single-chip microprocessors. Chip area is occupied by both core execution logic, such as ALU and FPU datapaths, and memory structures, such as caches, TLBs, and write buffers. This work focuses on the allocation of die area to memory structures through a cost/benefit analysis. The cost of memory structures with different sizes and associativities is estimated by using an established area model for on-chip memory. The performance benefits of selecting a given structure are measured through a collection of methods including on-the-fly hardware monitoring, trace-driven simulation and kernel-based analysis. Special consideration is given to operating systems that support multiple application programming interfaces (APIs), a software trend that substantially affects on-chip memory allocation decisions. Results: Small adjustments in cache and TLB design parameters can significantly impact overall performance. Operating systems that support multiple APIs, such as Mach 3.0, increase the relative importance of on-chip instruction caches and TLBs when compared against single-APl systems such as Ultrix.

Reduced instruction buffer for RISC architectures

External memory communication is one of the most limiting factors to processor performance. In this paper, a new mechanism is presented for RISC architectures and, in general, for those architectures with a fixed pipelined structure and data references generated by load/store instructions. The mechanism is based on the restricted use of and on-chip instruction buffer (RIB) to avoid those structural hazards generated by concurrent instruction/data memory accesses. An important area saving and performance improvement can be obtained when comparing with traditional on-chip instruction cache approaches.

Improving performance of small on-chip instruction caches

Most current single-chip processors employ an on-chip instruction cache to improve performance. A miss in this instruction cache will cause an external memory reference which must compete with data references for access to the external memory, thus affecting the overall performance of the processor. One common way to reduce the number of off-chip instruction requests is to increase the size of the on-chip cache. An alternative approach is presented in this paper, in which a combination of an instruction cache, instruction queue and instruction queue buffer is used to achieve the same effect with a much smaller instruction cache size. Such an approach is significant for emerging technologies where high circuit densities are initially difficult to achieve yet a high level of performance is desired, or for more mature technologies where chip area can be used to provide more functionality. The viability of this approach is demonstrated by its implementation in an existing single-chip processor.

Architectural and organizational tradeoffs in the design of the MultiTitan CPU

This paper describes the architectural and organizational tradeoffs made during the design of the MultiTitan, and provides data supporting the decisions made. These decisions covered the entire space of processor design, from the instruction set and virtual memory architecture through the pipeline and organization of the machine. In particular, some of the tradeoffs involved the use of an on-chip instruction cache with off-chip TLB and floating-point unit, the use of direct-mapped instead of associative caches, the use of 64-bit vs. 32-bit data bus, and the implementation of hardware pipeline interlocks.

A VLSI chip set for a multiprocessor workstation. I. An RISC microprocessor with coprocessor interface and support for symbolic processing

A 40-70 MIPS multiprocessor workstation is considered. VLSI implementation of the central processing unit (CPU) chip, based on reduced instruction set computer (RISC) architecture and with support for LISP, is described. The 1.3-cm/sup 2/ CPU chip uses a direct-mapped 512-b on-chip instruction cache and 138 40-b registers organized in eight overlapping windows to achieve 10 MIPS per processor peak performance with a 10-MHz, four-phase clock. >

On the design of on-chip instruction caches

Cache memories reduce memory latency and traffic in computing systems. Most existing caches are implemented as board-based systems. Advancing VLSI technology will soon permit significant caches to be integrated on chip with the processors they support. In designing on-chip caches, the constraints of VLSI become significant. The primary constraints are economic limitations on circuit area and off-chip communications. The paper explores the design of on-chip instruction-only caches in terms of these constraints. The primary contribution of this work is the development of a unified economic model of on-chip instruction-only cache design which integrates the points of view of the cache designer and of the floorplan architect. With suitable data, this model permits the rational allocation of constrained resources to the achievement of a desired cache performance. Specific conclusions are that random line replacement is superior to LRU replacement, due to an increased flexibility in VLSI floorplan design; that variable set associativity can be an effective tool in regulating a chip's floorplan; and that sectoring permits area efficient caches while avoiding high transfer widths. Results are reported on economic functionality, from chip area and transfer width to miss ratio. These results, or the underlying analysis, can be used by microprocessor architects to make intelligent decisions regarding appropriate cache organizations and resource allocations.

Dynamic RAM for on-chip instruction caches

On-chip memories are commonly used to reduce the external communication bandwidth of current microprocessors. Implementations with static RAM allow simple synchronization structures. The use of dynamic RAM results in an important area saving but complicates the processor control due to the refresh requirements. In this paper, a simple design approach, based on the use of dynamic RAM, is introduced for implementing on-chip instruction caches without complicating the processor control.

MIPS-X: a 20-MIPS peak, 32-bit microprocessor with on-chip cache

MIPS-X is a 32-b RISC microprocessor implemented in a conservative 2-/spl mu/m, two-level-metal, n-well CMOS technology. High performance is achieved by using a nonoverlapping two-phase 20-MHz clock and executing one instruction every cycle. To reduce its memory bandwidth requirements, MIPS-X includes a 2-kbyte on-chip instruction cache. The authors provide an overview of MIPS-X, focusing on the techniques used to reduce the complexity of the processor and implement the on-chip instruction cache.