Instruction-level Parallelism Research Articles

Ditching the Queue: Optimizing Coprocessor Utilization with Out-of-Order CPUs on Compact Systems on Chip

The growing demand for high-performance and energy-efficient processing in edge-oriented Systems-on-Chip is driving the adoption of dedicated integrated circuits that accelerate computationally intensive workloads. To minimize area and performance overhead, low-power, general-purpose CPUs are often tightly coupled with domain-specific coprocessors implementing custom instructions, thereby delivering higher throughput and reduced memory traffic. However, commonly used in-order CPUs are not optimized for instruction-level parallelism, leading to stalls in the instruction stream while waiting for long-latency coprocessor operations and under-utilization of the coprocessor while executing other instructions. This work investigates the benefits of replacing simple in-order cores with a more complex out-of-order architecture to dynamically schedule instructions for the main core and coprocessor, optimizing resource utilization and reducing execution time. To ensure generality, an in-depth analysis was carried out by offloading instructions to a custom dummy coprocessor capable of emulating iterative and pipelined operations with arbitrary latency. Various workloads simulating real-world applications were executed on two variants of an open-source microcontroller equipped with a recent out-of-order core and the state-of-the-art CV32E40X in-order core, respectively. Results from Register Transfer Level simulations show that the former configuration executes up to 60% more instructions per cycle, with a modest 12% system area overhead on a 65 nm CMOS technology node.

Improving performance of simultaneous multithreading CPUs using autonomous control of speculative traces

Simultaneous Multithreading (SMT) allows for a processor to concurrently execute multiple independent threads while sharing certain data path components to optimize resource waste. Speculative execution allows for these processors to take advantage of Instruction-Level Parallelism but the penalty for a miss speculation includes the wasting of resources amongst these shared resources where clock cycles are wasted at a time. In this paper we show that an average of 13 % of instructions are flushed as a result of incorrect predictions. These flushed out instructions could have potentially taken up shared resources which other non-speculative threads could have used. This paper proposes a technique that can dynamically adjust how many speculative instructions a thread can rename and decode aiming to diminish the waste of the shared resources. Our simulation results show, with the proposed technique, that the average flushed out instruction rate is reduced by 23 % and average throughput is improved by 13 %.

LUAEMA: A Loop Unrolling Approach Extending Memory Accessing for Vector Very-Long-Instruction-Word Digital Signal Processor with Multiple Register Files

Loop unrolling can provide more instruction-level parallelism opportunities for code and enables a greater range of instruction pipeline scheduling. In high-performance very-long-instruction-word (VLIW) digital signal processors (DSPs), there are special registers to address. To further improve the instruction-level parallelism of code for such DSPs by making full use of these registers, in this paper, we propose a more effective loop unrolling approach through extending memory accessing (LUAEMA). In this approach, the final unrolling factor is computed by a model in which every register kind and every memory accessing operation are considered. For basic digital signal processing algorithms, the unrolling factor under the LUAEMA is larger than that under the conventional loop unrolling approach. We also provide the opportunity to reduce the number of instructions in a loop during the code transformation of loop unrolling. The experimental results show that the loop unrolling approach proposed in this paper can achieve an average speedup ratio ranging from 1.14 to 1.81 compared with the conventional loop unrolling approach. For some algorithms, the peak speedup ratio is up to 2.11.

Instruction Level Parallelism and Memory Synchronization

The simultaneous or parallel execution of a series of instructions within a computer program is known as instruction-level parallelism, or ILP. ILP stands for the average number of instructions executed throughout each stage of this parallel execution, to be more precise. The architecture known as instruction level parallelism (ILP) allows for the execution of several operations in parallel within a single process, each with its own set of resources, including address space, registers, identifiers, state, and program counters. It describes compiler design strategies and processors intended to carry out operations in parallel to increase processor performance, such as memory load and store, integer addition, and float multiplication.

High-performance computing: Transitioning from Instruction-Level Parallelism to heterogeneous hybrid architectures

This paper delves into the shift from Instruction-Level Parallelism (ILP) to Heterogeneous Hybrid Parallel Computing in the quest for optimized performance processing. It sheds light on the constraints of ILP, emphasizing how these shortcomings have catalyzed a move toward the more adaptable and proficient framework of heterogeneous hybrid computing. This transformation's advantages are explored across diverse applications, notably in deep learning, cloud computing, data centers, and mobile SoCs. Additionally, the study underscores emerging architectures and innovations of this era, including many-core processors, FPGA-driven accelerators, and an assortment of software tools and libraries. While heterogeneous hybrid computing offers a promising horizon, it isn't without challenges. This paper brings to the fore issues like restricted adaptability, steep development costs, software compatibility hurdles, the absence of a standardized programming model, and vendor reliance. Through this in-depth exploration, our aim is to present a holistic snapshot of the present and potential future of high-performance processing.

Optimizing VLIW Instruction Scheduling via a Two-Dimensional Constrained Dynamic Programming

Typical embedded processors, such as Digital Signal Processors (DSPs), usually adopt Very Long Instruction Word (VLIW) architecture to improve computing efficiency. The performance of VLIW processors heavily relies on Instruction-Level Parallelism (ILP). Therefore, it is crucial to develop an efficient instruction scheduling algorithm to explore more ILP. While heuristic algorithms are widely used in modern compilers due to simple implementation and low computational cost, they have limitations in providing accurate solutions and are prone to local optima. On the other hand, exact algorithms can usually find the optimal solution, but their high time overhead makes them less suitable for large-scale problems. This paper proposes a two-dimensional constrained dynamic programming (TDCDP) approach and a quantitative model for instruction scheduling. The TDCDP approach achieves near-optimal solutions within an acceptable time overhead. Furthermore, we integrate our TDCDP approach into mainstream compiler architecture, encompassing Pre- and Post-RA (register allocation) scheduling. We conduct a quantitative evaluation of TDCDP compared to four heuristic algorithms on a typical VLIW processor. Our approach achieves an efficiency improvement of up to 58.34% in final solutions compared to the heuristic algorithms. Additionally, the Post-RA Scheduling enhances programs with an average speedup of 14.04% than solely applying the Pre-RA Scheduling.

Flip : Data-centric Edge CGRA Accelerator

Coarse-Grained Reconfigurable Arrays (CGRA) are promising edge accelerators due to the outstanding balance in flexibility, performance, and energy efficiency. Classic CGRAs statically map compute operations onto the processing elements (PE) and route the data dependencies among the operations through the Network-on-Chip. However, CGRAs are designed for fine-grained static instruction-level parallelism and struggle to accelerate applications with dynamic and irregular data-level parallelism, such as graph processing. To address this limitation, we present Flip , a novel accelerator that enhances traditional CGRA architectures to boost the performance of graph applications. Flip retains the classic CGRA execution model while introducing a special data-centric mode for efficient graph processing. Specifically, it leverages the inherent data parallelism of graph algorithms by mapping graph vertices onto PEs rather than the operations and supporting dynamic routing of temporary data according to the runtime evolution of the graph frontier. Experimental results demonstrate that Flip achieves up to 36× speedup with merely 19% more area compared to classic CGRAs. Compared to state-of-the-art large-scale graph processors, Flip has similar energy efficiency and 2.2× better area efficiency at a much-reduced power/area budget.

Investigating performance metrics for container-based HPC environments using x86 and OpenPOWER systems

Container-based High-Performance Computing (HPC) is changing the way computation is performed and reproduced without sacrificing the raw performance compared to hypervisor-assisted virtualization technologies. It primarily supports continuously evolving data-intensive applications such as computational fluid dynamics, seismic tomography, molecular biology, and Proteomics. OpenPOWER systems, unlike the x86 systems, use the POWER-compliant processor to exploit instruction-level and thread-level parallelism heavily. In our previous work, we designed and developed a Containerized HPC environment (cHPCe) from the scratch using Linux namespaces on OpenPOWER systems. This paper aims to provide an in-depth performance analysis of the Containerized HPC environment using x86 systems and Containerized HPC environment using the OpenPOWER system, on systems’ subcomponents, processor, memory, interconnect, and IO. This sub-component analysis provides an insight on several aspects of the system performance. To the best of our knowledge, no research has been reported yet for such a comparative analysis that designs cHPCe for both x86 and OpenPOWER systems. The performance of the developed cHPCe is compared with BareMetals, and VMs using the benchmarks HPCC, and IOZone. Our experimental results achieve 0.13% less compute performance penalty at its peak performance on cHPCe compared to the BareMetal-based solution for x86 systems. In contrast, a VM-based solution introduces an overhead of 20% and 4.83% in x86 and OpenPOWER cases, respectively. Moreover, the x86 and OpenPOWER systems observe inconsistent behavior for memory performance with a worst-case penalty of 9.68% and 6.64% compared to achieved peak performance, respectively. However, similar behavior is reported for cHPCe with an overhead of less than 3% and 2% in the worst case for the latency and bandwidth, respectively, compared to the BareMetal for network and disk performance. Our experimental results reveal that the containerized OpenPOWER environment represents a viable alternative to the counterpart x86 environment for the HPC solution.

On the interactions between ILP and TLP with hardware transactional memory

Hardware implementations of Transactional Memory (HTM) are designed to facilitate efficient thread synchronization in parallel programs, encouraging the use of larger critical sections. By employing optimistic concurrency control to execute transactions speculatively, HTM systems promise to deliver the performance benefits typically associated with fine-grained locks. In doing so, HTM systems must deal with transaction aborts. While under certain conditions aborts may be caused by the inherent limitations of hardware structures employed to implement TM (e.g., caches), conflicting concurrent accesses to shared memory locations are generally the prevailing cause for squashing the work done by a transactionIn this study, we present what we believe to be, to the best of our knowledge, the first characterization of how the aggressiveness of processor cores, particularly their ability to exploit instruction-level parallelism (ILP), interacts with the support for optimistic thread-level speculation offered by HTM systems. We have observed that by adjusting the size of structures that facilitate out-of-order and speculative execution, the number of aborts in the execution of transactional workloads can be altered in best-effort HTM implementations. Our findings indicate that in scenarios with high contention, a smaller number of powerful cores is more suitable, whereas in low contention scenarios, using a larger number of less aggressive cores is preferable. In addition, HTM systems that employ lazy detection and those employing eager detection with requester-stalls resolution, benefit from using simpler cores. In conclusion, abort ratios can be reduced with a careful choice of both processor aggressiveness and design aspects for each application depending on its contention.

RenderBench: The CPU Rendering Benchmark Suite Based on Microarchitecture-Independent Characteristics

This research addresses the issue of evaluating CPU rendering performance by introducing the innovative benchmark test suite construction method RenderBench. This method combines CPU microarchitecture features with rendering task characteristics to comprehensively assess CPU performance across various rendering tasks. Adhering to principles of representativeness and comprehensiveness, the constructed benchmark test suite encompasses diverse rendering tasks and scenarios, ensuring accurate capture of CPU performance features. Through data sampling and in-depth analysis, this study focuses on the role of microarchitecture-independent features in rendering programs, including instruction-level parallelism, instruction mix, branch prediction capability, register dependency distance, data flow stride, and memory reuse distance. The research findings reveal significant variations in rendering programs across these features. For instance, in terms of instruction-level parallelism, rendering programs demonstrate a high level of ILP (instruction-level parallelism), with an average value of 5.70 for ILP256, surpassing benchmarks such as Mibench and NAS Parallel Benchmark. Furthermore, in aspects such as instruction mix, branch prediction capability, register dependency distance, data flow stride, and memory reuse distance, rendering programs exhibit distinct characteristics. Through the application of the RenderBench method, a scalable and highly representative benchmark test suite was constructed, facilitating an in-depth exploration of CPU performance bottlenecks in rendering tasks. By delving into microarchitecture-independent features, this study provides profound insights into rendering program performance, offering valuable guidance for optimizing CPU rendering performance. The application of ensemble learning models, such as random forest, XGBoost, and ExtraTrees, reveals the significant influence of features like floating-point computation, memory access patterns, and register usage on CPU rendering program performance. These insights not only offer robust guidance for performance optimization but also underscore the importance of feature selection and algorithm choice. In summary, the results of feature importance ranking in this study provide beneficial directions and deep insights for the optimization and enhancement of CPU rendering program performance. These findings are poised to exert a positive impact on future research and development endeavors.

Consistency Constraints for Mapping Dataflow Graphs to Hybrid Dataflow/von Neumann Architectures

Dataflow process networks (DPNs) provide a convenient model of computation that is often used to model system behavior in model-based designs. With fixed sets of nodes, they are also used as dataflow graphs as an intermediate program representation by compilers to uncover instruction-level parallelism of sequential programs. Many recent processor architectures, which are still von Neumann architectures, also use dataflow computing to increase their exploitation of instruction-level parallelism by exposing their datapaths so that the compiler can take care of the allocation of processing units (PUs), the execution schedules of instructions on the PUs, and the communication of intermediate values between PUs. If the communication paths are buffered, these architectures can be abstracted into a DPN architecture whose PUs and interconnection network are DPN nodes. In this article, we introduce a DPN abstraction of hybrid dataflow/von Neumann architectures and consider the mapping of the nodes of a given dataflow graph to the PUs of such a DPN architecture such that there are no conflicts due to the mapping of different nodes to the same PU. We express the allocation and scheduling constraints in terms of propositional logic for the original dataflow graph and for a modified version of the dataflow graph that simplifies the constraints by introducing levels using copy nodes, such that all nodes receive inputs only from nodes of the previous level. We also formulate equisatisfiable SMT constraints using integer variables to reason directly about the parallel runtime. On this basis, we further present alternative SAT constraints that explicitly encode concurrency, and discuss variants of the constraints for a better understanding of the same.

MaskSIMD-lib: on the performance gap of a generic C optimized assembly and wide vector extensions for masked software with an Ascon-p test case

Efficient implementations of software masked designs constitute both an important goal and a significant challenge to Side Channel Analysis attack (SCA) security. In this paper we discuss the shortfall between generic C implementations and optimized (inline-) assembly versions while providing a large spectrum of efficient and generic masked implementations for any order, and demonstrate cryptographic algorithms and masking gadgets with reference to the state of the art. Our main goal is to show the prime performance gaps we can expect between different implementations and suggest how to harness the underlying hardware efficiently, a daunting task for various masking-orders or masking algorithm (multiplications, refreshing etc.). This paper focuses on implementations targeting wide vector bitsliced designs, such as the ISAP algorithm. We explore concrete instances of implementations utilizing processors enabled by wide-vector capability extensions of the AMD64 Instruction Set Architecture (ISA); namely, the SSE2/3/4.1, AVX-2 and AVX-512 Streaming Single Instruction Multiple Data extensions. These extensions mainly enable efficient memory level parallelism and provide a gradual reduction in computation-time as a function of the level of extension and the hardware support for instruction-level parallelism. For the first time we provide a complete open-source repository of such gadgets tailored for these extensions, various gadgets types and for all orders. We evaluate the disparities between generic high-level language masking implementations for optimized (inline-) assembly and conventional single execution path data-path architectures such as the ARM architecture. We underscore the crucial trade-off between state storage in the data-memory as compared to keeping it in the register-file (RF). This relates specifically to masked designs, and is particularly difficult to resolve because it requires inline-assembly manipulations and is not natively supported by compilers. Moreover, as the masking order (d) increases and the state gets larger, there must be an increase in data memory read/write accesses for state handling since the RF is simply not large enough. This requires careful optimization which depends to a considerable extent on the underlying algorithm to implement. We discuss how full utilization of SSE extensions is not always possible; i.e. when d is not a power of two, and pin-point the optimal d values and very sub-optimal values of d which aggressively under-utilize the hardware. More generally, this paper presents several different fully generic masked implementations for any order or multiple highly optimized (inline-) assembly instances which are quite generic (for a wide spectrum of ISAs and extensions), and provide very specific implementations targeting specific extensions. The goal is to promote open-source availability, research, improvement and implementations relating to SCA security and masked designs. The building blocks and methodologies provided here are portable and can be easily adapted to other algorithms.

Software Pipelining for Quantum Loop Programs

We propose a method for performing software pipelining on quantum for-loop programs, exploiting parallelism in and across iterations. We redefine concepts that are useful in program optimization, including array aliasing, instruction dependency and resource conflict, this time in optimization of quantum programs. Using the redefined concepts, we present a software pipelining algorithm exploiting instruction-level parallelism in quantum loop programs. The optimization method is then evaluated on some test cases, including popular applications like QAOA, and compared with several baseline results. The evaluation results show that our approach outperforms loop optimizers exploiting only in-loop optimization chances by reducing total depth of the loop program to close to the optimal program depth obtained by full loop unrolling, while generating much smaller code in size. This is the first step towards optimization of a quantum program with such loop control flow as far as we know.

Optimal placement of PMU for enhancing power system monitoring

Phasor Measurement Units (PMUs) have emerged as crucial tools in the field of power system monitoring, control, and safety. Extensive research has been conducted on enhancing the capabilities of PMUs. Traditional SCADA techniques, which have been employed for many years, gather data on power system parameters at intervals of 4 to 10 seconds, offering a static perspective of the system's behavior. However, relying solely on static data may limit the effectiveness of the monitoring and control. On the other hand, PMUs provide real-time information about the dynamic state of the system, enabling more comprehensive insights. For the optimal and cost-effective installation of PMUs, it is vital to strategically position them in locations that effectively monitor the state of the system. The Instruction Level Parallelism (ILP) approach was employed to identify the ideal PMU placement for power system observability. The ILP approach determines the most suitable positions for PMUs by considering factors such as the redundancy of measurements and cost efficiency. To validate the effectiveness of the ILP approach, it was tested using the widely recognized IEEE 14 bus system, which serves as a standard benchmark for power-system analysis. Through the ILP approach, the optimal placement of the PMUs can be determined, thereby enhancing the observability of the power system. This in turn improves situational awareness, enables effective fault detection, and facilitates coordinated control strategies. By integrating PMUs and the ILP approach, the power system monitoring and control can be significantly enhanced. Ongoing research and development in PMU technology and placement methodologies continues to advance the field, enabling more efficient and effective management of modern power grids.

SpecBox: A Label-Based Transparent Speculation Scheme Against Transient Execution Attacks

Speculative execution techniques have been a cornerstone of modern processors to improve instruction-level parallelism. However, recent studies showed that this kind of techniques could be exploited by attackers to leak secret data via transient execution attacks, such as Spectre. Many defenses are proposed to address this problem, but they all face various challenges: (1) Tracking data flow in the instruction pipeline could comprehensively address this problem, but it could cause pipeline stalls and incur high performance overhead; (2) Making side effect of speculative execution imperceptible to attackers, but it often needs additional storage components and complicated data movement operations. In this article, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">label-based transparent speculation</i> scheme called <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SpecBox</small> . It dynamically partitions the cache system to isolate speculative data and non-speculative data, which can prevent transient execution from being observed by subsequent execution. Moreover, it uses thread ownership semaphores to prevent speculative data from being accessed across cores. In addition, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SpecBox</small> also enhances the auxiliary components in the cache system against transient execution attacks, such as hardware prefetcher. Our security analysis shows that <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SpecBox</small> is secure and the performance evaluation shows that the performance overhead on SPEC CPU 2006 and PARSEC-3.0 benchmarks is small.

Simulation of Pipelined MIPS Floating-Point Units using Node-RED

The pipelined processor architecture is the best way to increase instruction-level parallelism, and thus, understanding its operation is one of the keys in computer architecture learning. To help with the leaning process, we have devised a series of pipeline simulation methodologies. This article presents one of them – a simulation methodology of hazard detection and forwarding in MIPS32 pipelined floating-point units. Our implementation approach is using the Node-RED programming environment, an event-driven dragand-drop system for designing data flows with business logic. In addition, to implement sophisticated operations not supported by Node-RED, we also employ WebAssembly code and the Rust language. We simulate not only the standard 5-stage pipelined MIPS ISA but also pipelined (addition, subtraction, and multiplication) and unpipelined (division) floating-point operations. Our study focuses mainly on hazard detection, which is required to ensure program correctness, as well as forwarding used to improve system performance. Lastly, we design a dashboard interface to visually represent the pipeline stages and CPU status during execution. Using the dashboard interface, MIPS32 machine code can be loaded into our simulator from hexadecimal text files. We verified that our simulator is handling hazard detection and forwarding correctly. A screenshot of the dashboard interface is included that shows all the stages of floating-point pipelines.

A Survey on Memory-centric Computer Architectures

Faster and cheaper computers have been constantly demanding technological and architectural improvements. However, current technology is suffering from three technology walls: leakage wall, reliability wall, and cost wall. Meanwhile, existing architecture performance is also saturating due to three well-known architecture walls: memory wall, power wall, and instruction-level parallelism (ILP) wall. Hence, a lot of novel technologies and architectures have been introduced and developed intensively. Our previous work has presented a comprehensive classification and broad overview of memory-centric computer architectures. In this article, we aim to discuss the most important classes of memory-centric architectures thoroughly and evaluate their advantages and disadvantages. Moreover, for each class, the article provides a comprehensive survey on memory-centric architectures available in the literature.

G-RMOS: GPU-accelerated Riemannian Metric Optimization on Surfaces

Surface mapping is used in various brain imaging studies, such as for mapping gray matter atrophy patterns in Alzheimer’s disease. Riemannian metrics on surface (RMOS) is a state-of-the-art surface mapping algorithm that optimizes Riemannian metrics to establish one-to-one correspondences between surfaces in the Laplace–Beltrami embedding space. However, owing to the complex calculation with accurate one-to-one correspondences, RMOS registration takes a long time. In this study, we propose G-RMOS, a graphics processing unit (GPU)-accelerated RMOS registration pipeline that uses three GPU kernel design strategies: 1. using GPU computing capability with a batch scheme; 2. using the cache in the GPU block to minimize memory latency in register and shared memory; and 3. maximizing the effective number of instructions per GPU cycle using instruction level parallelism. Using the experimental results, we compare the acceleration speed of the G-RMOS framework with that of RMOS using hippocampus and cortical surfaces, and show that G-RMOS achieves a significant speedup in surface mapping. We also compare the memory requirements for cortical surface mapping and show that G-RMOS uses less memory than RMOS.

Toward Accurate and Fast Summation

We introduce a new accurate summation algorithm based on the error-free summation into floating-point buckets. Our algorithm exploits ideas from Zhu and Hayes’ OnlineExactSum , but it uses a significantly smaller number of accumulators and has a better instruction-level parallelism. In the default setting, our implementation aaaSum returns a faithfully rounded floating-point approximation of the true sum. We also discuss possible modifications for the computation of reproducible, correctly rounded, and multiple precision floating-point approximations. The computational overhead for any of these modifications is kept comparably small. Numerical tests demonstrate that aaaSum performs well for very small to large problem sizes, independent of the condition number of the problem. We compare our algorithm with other accurate and high-precision summation approaches.

An Application Specific Vector Processor for Efficient Massive MIMO Processing

This paper presents an implementation for a baseband massive multiple-input multiple-output (MIMO) application-specific instruction set processor (ASIP). The ASIP is geared with vector processing capabilities in the form of single instruction multiple data (SIMD), and furthermore exploits instruction level parallelism by employing a very large instruction word (VLIW) architecture. Additionally, a systolic array is built into the pipeline which is tuned to speed up matrix calculations. A parallel memory subsystem and stand-alone accelerators are integrated into the ASIP architecture in order to meet the processing requirement. The processor is synthesized in 22 nm FD-SOI technology running at a clock frequency of 800 MHz. The system achieves a maximum detection throughput of 0.75 Gb/s/mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> for a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$128\times 8$ </tex-math></inline-formula> massive MIMO system.