Cycle-accurate Simulator Research Articles

FPGA friendly NoC simulation acceleration framework employing the hard blocks

A major role is played by Modeling and Simulation platforms in development of a new Network-on-Chip (NoC) architecture. The cycle accurate software simulators tend to become slow when simulating thousands of cores on a single chip. FPGAs have become the vehicle for simulation acceleration due to the properties of parallelism. Most of the state-of-the-art FPGA based NoC simulators utilize soft logic only for modeling the NoCs, leaving out the hard blocks to be unutilized. In this work, the FIFO Buffer and Crossbar switch functionalities of the NoC router have been embedded in the Block RAM (BRAMs) and the DSP48E1 slices with large multiplexer respectively. Employing the proposed techniques of mapping the NoC router components on the FPGA hard blocks, an NoC simulation acceleration framework based on the FPGA is presented in this work. A huge reduction in the use of the Configurable Logic Blocks (CLBs) has been observed when the FIFO buffer and Crossbar components of the NoC topology’s router micro-architecture are embedded in FPGA hard blocks. Our experimental results show that the topologies implemented employing the proposed FPGA friendly mapping of the NoC router components on the hard blocks consume 43.47% fewer LUTs and 41.66% fewer FFs than the topologies with CLB implementation. To optimize the latency of the NoC under consideration, a control unit called “buf_empty_checker” has been employed. A reduction in average latency has been observed compared to the CLB based topology implementation employing the proposed mapping. The proposed work consumes 10.88% fewer LUTs than the CONNECT NoC generation tool. Compared to DART, a reduction of 73.38% and 66.55% in LUTs and FFs has been observed with respect to the proposed work. The average packet latency of the proposed NoC architecture is 24.8% and 19.1% lesser than the CONNECT and DART architectures.

Sparse-PE: A Performance-Efficient Processing Engine Core for Sparse Convolutional Neural Networks

Sparse convolutional neural network (CNN) models reduce the massive compute and memory bandwidth requirements inherently present in dense CNNs without a significant loss in accuracy. Sparse CNNs, however, present their own set of challenges including non-linear data accesses and complex design of CNN processing elements (PEs). Recently proposed accelerators like SCNN, Eyeriss v2, and SparTen, exploit the two-sided sparsity, that is, sparsity in both the input activations and weights to accelerate the CNN inference. These, accelerators, however, suffer from a multitude of problems that limit their applicability, such as inefficient micro-architecture (SCNN, Eyeriss v2), complex PE design (Eyeriss v2), no support for non-unit stride convolutions (SCNN) and FC layers (SparTen, SCNN). To address these issues in contemporary sparse CNN accelerators, we propose Sparse-PE , a multi-threaded, and flexible CNN PE, capable of handling both the dense and sparse CNNs. The Sparse-PE core uses binary mask representation and actively skips computations involving zeros and favors non-zero computations, thereby, drastically increasing the effective throughput and hardware utilization. Unlike previous designs, the Sparse-PE core is generic in nature and not targeted towards a specific accelerator, and thus, can also be used as a standalone sparse dot product compute engine. We evaluate the performance of the core using a custom built cycle accurate simulator. Our simulations show that the Sparse-PE core-based accelerator provides a performance gain of $12\times $ over a recently proposed dense accelerator (NeuroMAX). For sparse accelerators, it provides a performance gain of $4.2\times $ , $2.38\times $ , and $1.98\times $ over SCNN, Eyeriss v2, and SparTen, respectively.

3RSeT: Read Disturbance Rate Reduction in STT-MRAM Caches by Selective Tag Comparison

Recent development in memory technologies has introduced Spin-Transfer Torque Magnetic RAM (STT-MRAM) as the most promising replacement for SRAMs in on-chip cache memories. Besides its lower leakage power, higher density, immunity to radiation-induced particles, and non-volatility, an unintentional bit flip during read operation, referred to as read disturbance error, is a severe reliability challenge in STT-MRAM caches. One major source of read disturbance error in STT-MRAM caches is simultaneous accesses to all tags for parallel comparison operation in a cache set, which has not been addressed in previous work. This paper first demonstrates that high read accesses to tag arrays extremely increase the read disturbance rate and then proposes a low-cost scheme, so-called Read Disturbance Rate Reduction in STT-MRAM Caches by Selective Tag Comparison (3RSeT), to reduce the error rate by eliminating a significant portion of tag reads. 3RSeT proactively disables the tags that have no chance for hit, using low significant bits of the tags on each access request. Our evaluations using gem5 full-system cycle-accurate simulator show that 3RSeT reduces the read disturbance rate in the tag array by 71.8%, which results in 3.6x improvement in Mean Time To Failure. In addition, the energy consumption is reduced by 62.1% without compromising performance and with less than 0.4% area overhead.

Efficient Nearest-Neighbor Data Sharing in GPUs

Stencil codes (a.k.a. nearest-neighbor computations) are widely used in image processing, machine learning, and scientific applications. Stencil codes incur nearest-neighbor data exchange because the value of each point in the structured grid is calculated as a function of its value and the values of a subset of its nearest-neighbor points. When running on Graphics Processing Unit (GPUs), stencil codes exhibit a high degree of data sharing between nearest-neighbor threads. Sharing is typically implemented through shared memories, shuffle instructions, and on-chip caches and often incurs performance overheads due to the redundancy in memory accesses. In this article, we propose Neighbor Data (NeDa), a direct nearest-neighbor data sharing mechanism that uses two registers embedded in each streaming processor (SP) that can be accessed by nearest-neighbor SP cores. The registers are compiler-allocated and serve as a data exchange mechanism to eliminate nearest-neighbor shared accesses. NeDa is embedded carefully with local wires between SP cores so as to minimize the impact on density. We place and route NeDa in an open-source GPU and show a small area overhead of 1.3%. The cycle-accurate simulation indicates an average performance improvement of 21.8% and power reduction of up to 18.3% for stencil codes in General-Purpose Graphics Processing Unit (GPGPU) standard benchmark suites. We show that NeDa’s performance is within 13.2% of an ideal GPU with no overhead for nearest-neighbor data exchange.

FLASH: Fast, Parallel, and Accurate Simulator for HLS

A large semantic gap between a high-level synthesis (HLS) design and a low-level RTL simulation environment often creates a barrier for those who are not field-programmable gate array (FPGA) experts. Moreover, such a low-level simulation takes a long time to complete. Software HLS simulators can help bridge this gap and accelerate the simulation process; but their shortcoming is that they do not provide performance estimation. To make matters worse, we found that the current FPGA HLS commercial software simulators sometimes produce incorrect results. In order to solve these performance estimation and correctness problems while maintaining the high speed of software simulators, this article proposes a new HLS simulation flow named FLASH. The main idea behind the proposed flow is to extract scheduling information from the HLS tool and automatically construct an equivalent cycle-accurate simulation model while preserving C semantics. The experimental results show that FLASH runs three orders of magnitude faster than the RTL simulation.

A Bandwidth Control Arbitration for SoC Interconnections Performing Applications with Task Dependencies.

Current System-on-Chips (SoCs) execute applications with task dependency that compete for shared resources such as buses, memories, and accelerators. In such a structure, the arbitration policy becomes a critical part of the system to guarantee access and bandwidth suitable for the competing applications. Some strategies proposed in the literature to cope with these issues are Round-Robin, Weighted Round-Robin, Lottery, Time Division Access Multiplexing (TDMA), and combinations. However, a fine-grained bandwidth control arbitration policy is missing from the literature. We propose an innovative arbitration policy based on opportunistic access and a supervised utilization of the bus in terms of transmitted flits (transmission units) that settle the access and fine-grained control. In our proposal, every competing element has a budget. Opportunistic access grants the bus to request even if the component has spent all its flits. Supervised debt accounts a record for every transmitted flit when it has no flits to spend. Our proposal applies to interconnection systems such as buses, switches, and routers. The presented approach achieves deadlock-free behavior even with task dependency applications in the scenarios analyzed through cycle-accurate simulation models. The synergy between opportunistic and supervised debt techniques outperforms Lottery, TDMA, and Weighted Round-Robin in terms of bandwidth control in the experimental studies performed.

Toward a general framework for jointly processor-workload empirical modeling

The complexity of state-of-the-art processor architectures and their consequent vast design spaces have made it difficult and time-consuming to explore the best configuration for them. Design space exploration (DSE) refers to systematic analysis and pruning of unwanted design points based on parameters of interest. DSE requires analysis and estimation of performance criteria of design points. A more accurate estimation produces a more efficient target design. A typical estimation method is machine learning approaches based on statistical inference, also known as empirical modeling, which requires only a limited number of simulations. Undoubtedly, an empirical model finds the optima much faster than using cycle-accurate simulations and is much more accurate than employing analytical models. For that purpose, our paper proposes a general methodology and a framework to find an appropriate and most accurate empirical model to estimate the performance of general-purpose or embedded multiprocessors running multithreaded workloads. This framework consists of three main steps: (1) Workload characterization and clustering, (2) Finding optimal model, and (3) Estimating the performance of a new workload outside the training set. These optimal performance prediction models could be utilized in the process of exploring the architectural design space. An experimental case is also tested using this framework for feasibility purposes. Validation experiments show MAEs less than 10% for this case.

SDAM: a combined stack distance-analytical modeling approach to estimate memory performance in GPUs

Graphics processing units (GPUs) are powerful in performing data-parallel applications. Such applications most often rely on the GPU’s memory hierarchy to deliver high performance. Designing efficient memory hierarchy for GPUs is a challenging task because of its wide architectural space. To moderate this challenge, this paper proposes a framework, called stack distance-analytic modeling (SDAM), to estimate memory performance of the GPU in terms of memory cycle counts. Providing the input data to the model is crucial in terms of the accuracy of the input data, and the time spent to obtain them. SDAM employs the stack distance analysis method and analytical modeling to obtain the required input accurately and swiftly. Further, it employs a detailed analytical model to estimate memory cycles. SDAM is validated against real GPU executions. Further, it is compared with a cycle accurate simulator. The experimental evaluations, performed on a set of memory-intensive benchmarks, prove that SDAM is faster and more accurate than cycle-accurate simulation, thus it can facilitate the GPU cache design-space exploration. For a selection of data-intensive benchmarks, SDAM showed a 32% average error in estimating memory data transfer cycles in a modern GPU, which outperforms cycle-accurate simulation, while it is an order of magnitude faster than the cycle-accurate simulation. Finally, the applicability of SDAM in exploring cache design-space in GPUs is demonstrated through experimenting with various cache designs.

Learning-Based Quality Management for Approximate Communication in Network-on-Chips

Current multi/many-core systems spend large amounts of time and power transmitting data across on-chip interconnects. This problem is aggravated when data-intensive applications, such as machine learning and pattern recognition, are executed in these systems. Recent studies show that some data-intensive applications can tolerate modest errors, thus opening a new design dimension, namely, trading result quality for better system performance. In this article, we explore application error tolerance and propose an approximate communication framework to reduce the power consumption and latency of network-on-chips (NoCs). The proposed framework incorporates a quality control method and a data approximation mechanism to reduce the packet size to decrease network power consumption and latency. The quality control method automatically identifies the error-resilient variables that can be approximated during transmission and calculates their error thresholds based on the quality requirements of the application by analyzing the source code. The data approximation method includes a lightweight lossy compression scheme, which significantly reduces packet size when the error-resilient variables are transmitted. This framework results in fewer flits in each data packet and reduces traffic in NoCs while guaranteeing the quality requirements of applications. Our cycle-accurate simulation using the AxBench benchmark suite shows that the proposed approximate communication framework achieves 62% latency reduction and 43% dynamic power reduction compared to previous approximate communication techniques while ensuring 95% result quality.

An Asymmetric, One-To-Many Traffic-Aware mm-Wave Wireless Interconnection Architecture for Multichip Systems

Platform based computing modules such as embedded systems and micro-servers are multichip systems with in-package memory and processing chips. Such systems consist of both one-to-one (unicast) and one-to-many (broadcast/ multicast) traffic patterns. State-of-the-art wired interconnection architectures such as Network-on-Chip (NoC) are specially designed to handle on-chip unicast traffic and cannot mask the high off-chip communication latency caused by the chip-to-chip I/Os in multichip systems. Moreover, with the increase in memory-intensive applications and hence one-to-many traffic in a multichip system, this scenario gets even worse as conventionally one-to-many traffic is handled as multiple unicast traffic in a multihop NoC infrastructure. Consequently, a small proportion of such one-to-many traffic increases energy consumption and message latency significantly for chip-to-chip communication. Therefore, to support such one-to-many traffic, these multichip systems with in-package memory need one-to-many traffic-aware interconnection infrastructure. To address these issues, we propose the design of one-to-many traffic-aware Wireless Network-in-Package (WiNiP) architecture by using a novel asymmetric WiNiP topology and a traffic-aware Medium Access Control (MAC) mechanism. With cycle-accurate simulations, we demonstrate that the proposed WiNiP architecture reduces the energy consumption and latency up to 46.96 and 47.08 percent respectively for multichip data transfer compared to state-of-the-art wired NiPs for application-specific traffic.

Semi-automatic validation of cycle-accurate simulation infrastructures: The case for gem5-x86

Since the early 70s, simulation infrastructures have been a keystone in computer architecture research, providing a fast and reliable way to prototype and evaluate ideas for future computing systems. There are different types of simulators, from most detailed (cycle-accurate) to time-based/functional and analytical modeling. Increasing accuracy translates into several orders of magnitude in terms of simulation speed. Yet, a question remains open: are the results derived from the simulation infrastructure representative of a real machine?Validation of these infrastructures is complex and costly, usually performed upon release. However, most simulators do not provide the appropriate means to verify or validate new architectural models. In this paper, we introduce a semi-automatic validation framework based on real-hardware performance counter information. The framework provides two levels of abstraction: (a) a high level definition of the processor behavior (Top-Down model) and (b) detailed per-structure and per-pipeline-stage usage breakdown to pinpoint simulator issues. We used this framework to validate the latest available gem5-x86 simulation environment, and found several sources of error that alter the expected behavior of the simulated processor, which we were later to document and correct.

An Approximate Communication Framework for Network-on-Chips

Current multi-/many-core systems spend large amounts of time and power transmitting data across on-chip interconnects. This problem is aggravated when data-intensive applications, such as machine learning and pattern recognition, are executed in these systems. Recent studies show that some data-intensive applications can tolerate modest errors, thus opening a new design dimension, namely, trading result quality for better system performance. In this article, we explore application error tolerance and propose an approximate communication framework to reduce the power consumption and latency of network-on-chips (NoCs). The proposed framework incorporates a quality control method and a data approximation mechanism to reduce the packet size to decrease network power consumption and latency. The quality control method automatically identifies the error-resilient variables that can be approximated during transmission and calculates their error thresholds based on the quality requirements of the application by analyzing the source code. The data approximation method includes a lightweight lossy compression scheme, which significantly reduces packet size when the error-resilient variables are transmitted. This framework results in fewer flits in each data packet and reduces traffic in NoCs while guaranteeing the quality requirements of applications. Our cycle-accurate simulation using the AxBench benchmark suite shows that the proposed approximate communication framework achieves 62 percent latency reduction and 43 percent dynamic power reduction compared to previous approximate communication techniques while ensuring 95 percent result quality.

TAMER: an adaptive task allocation method for aging reduction in multi-core embedded real-time systems

Technology scaling has exacerbated the aging impact on the performance and reliability of integrated circuits. By entering into nanotechnology era in recent years, the power density per unit of area has increased, which leads to a higher chip temperature. Aging in a chip is originated from multiple phenomena; all of them are intensified by increased temperature. Several circuit- and architecture-level schemes tried to mitigate the aging in the literature. However, these schemes are not sufficient for multi-core systems due to their unawareness of the unique constraints and features of these platforms. In this paper, we propose a system-level aging mitigation method, so-called Adaptive Task Allocation for Aging Reduction in Multi-core Embedded Real-time Systems (TAMER). As a task allocation algorithm, TAMER takes the cores’ utilization and their internal units’ activity into account to smooth the temperature pattern inside the chip. By minimizing both temporal and spatial thermal variations, TAMER prevents the occurrence of hotspot over time and space. We evaluated the TAMER method using a framework consisting of gem5 full-system cycle-accurate simulator, MATLAB, ESESC multi-core simulator, and HotSpot temperature modeling tool. The simulation results show that TAMER decreases the maximum and average temperature standard deviation of the cores by 56% and 37%, respectively, compared to the best previous temperature distribution task allocation algorithm. It is worth mentioning that, neither area nor performance overhead has been imposed on the system after the aforementioned improvements.

Experiment and enabled flow for GPGPU-Sim simulators with fixed-point instructions

Abstract Currently, GPGPU-Sim has become an important vehicle for academic architecture research. It is a cycle-accurate simulator that models the contemporary graphics processing unit. Machine learning has now been widely used in various applications such as self-driving car, mobile devices, and medication. With the popularity of mobile devices, mobile vendors are interested on porting machine learning or deep learning applications from computers to mobile devices. Google has developed TensorFlow Lite and Android NNAPI for mobile and embedded devices. Since machine learning and deep learning are very computationally intensive, the energy consumption has become a serious problem in mobile devices. Moreover, Moore’s law cannot last forever. Hence, the performance of the mobile device and computers such as desktops or servers will have limited enhancements in the foreseeable future. Therefore, the performance and the energy consumption are two issues of great concern. In this paper, we proposed a new data type, fixed-point, which is a low-power numerical data type that can reduce energy consumption and enhance performance in machine learning applications. We implemented the fixed-point instructions in the GPGPU-Sim simulator and observed the energy consumption and performance. Our evaluation demonstrates that by using the fixed-point instructions, the proposed design exhibits improved energy savings. Our experiment indicate that the use of fixed-point data type saves at least 14% of total GPU energy consumption than floating-point data type.

A study on the performance impact of programmable logic controllers based on enhanced architecture and organization

Since their appearance, programmable logic controllers (PLCs) are massively and predominantly used as the central controller in automation systems. Unfortunately, due to the poor performance of the majority of these devices, the typical role of PLCs in automation systems becomes restricted to a simple controller, since applications with more sophisticated computational requirements tend to be handled by external processing units along with the PLCs. To solve this issue, this work improves novel architecture proposals based on data flow machines, circuit simulation theory, and the memoization technique to achieve a performance boost based on the scan time reduction. Along with the architectural improvements, this paper evaluates the impact of different execution units’ types and quantities in a cycle-accurate simulator (CAS) that was specially developed to simulate the PLC cores. Furthermore, in order to perform a robust and complete evaluation, the silicon areas of the simulated architectures are calculated using the McPAT framework to establish the performance/area relationship of the simulated cores. Evaluation results show best scan time reductions of up to 68% for cores with single execution units and up to 89% for cores with multiple execution units, as well as a best-case of 50% scan time reduction with an acceptable impact on the silicon area. Lastly, the evaluation of the results of the proposed improved cores with multiple execution units shows that they outperform the theoretical performance limit of multiple execution units based on Amdahl’s law up to 4 execution units.

Fast modeling L2 cache reuse distance histograms using combined locality information from software traces

Abstract To mitigate the performance gap between CPU and the main memory, multi-level cache architectures are widely used in modern processors. Therefore, modeling the behaviors of the downstream caches becomes a critical part of the processor performance evaluation in the early stage of Design Space Exploration (DSE). In this paper, we propose a fast and accurate L2 cache reuse distance histogram model, which can be used to predict the behaviors of the multi-level cache architectures where the L1 cache uses the LRU replacement policy and the L2 cache uses LRU/Random replacement policies. We use the profiled L1 reuse distance histogram and two newly proposed metrics, namely the RST table and the Hit-RDH, that describing more detailed information of the software traces as the inputs. For a given L1 cache configuration, the profiling results can be reused for different configurations of the L2 cache. The output of our model is the L2 cache reuse distance histogram, based on which the L2 cache miss rates can be evaluated. We compare the L2 cache miss rates with the results from gem5 cycle-accurate simulations of 15 benchmarks chosen from SPEC CPU 2006 and 9 benchmarks from SPEC CPU 2017. The average absolute error is less than 5%, while the evaluation time for each L2 configuration can be sped up almost 30× for four L2 cache candidates.

A one-to-many traffic-oriented mm-wave wireless network-in-package interconnection architecture for multichip computing systems

High Performance Computing (HPC) platforms like blade servers consist of multiple processor chips, which may be multicore CPUs, GPUs, memory modules and other subsystems. In such memory and computation intensive systems one-to-many traffic patterns originate from cache coherency, system-level synchronization mechanisms and other control signals. However, small portions of such traffic can introduce huge local and global congestion, which significantly reduce overall performance and cause energy bottleneck unless low latency transmission is ensured by a one-to-many traffic-aware interconnection architecture. Therefore, these high performance and memory intensive multichip systems require efficient support for one-to-may traffic. Traditional metal-based Network-on-Chip (NoC) interconnection architecture is mostly designed for unicast traffic and therefore not suitable for such one-to-many traffic as it provides high-latency, power-hungry multi-hop paths. To address this issue, we propose the design of a one-to-many traffic-aware Wireless Network-in-Package (WiNiP) architecture by integrating a novel asymmetric wireless interconnection topology, a novel hybrid two-state Medium Access Control (MAC) and flow control. The proposed asymmetric topology, MAC and flow control collaborate with each other to form a low latency, one-to-many traffic-aware WiNiP interconnection architecture and increase system bandwidth with lower energy consumption. Through cycle-accurate simulator we show that the proposed topology reduces average packet latency by 47.92 % and outperforms other interconnection architectures for on and off-chip data transfer for synthetic as well as application-specific traffic patterns.

Scalable and energy efficient wireless inter chip interconnection fabrics using THz-band antennas

Computing platforms ranging from embedded systems to server blades comprise of multiple Systems-on-Chips (SoCs). Conventionally, communication between chips in these multichip platforms are realized using high-speed I/O modules over metal traces on a substrate. Due to the high-power consumption of I/O modules and non-scalable pitch of pins or solder bumps their bandwidth density and power consumption becomes bottleneck for multichip systems. Wireless chip-to-chip communication is emerging as an alternative solution to the traditional interconnection challenges of multichip systems. Novel devices based on graphene structures capable of establishing wireless links are explored in recent literature to provide high bandwidth THz links. In this work, we propose to utilize graphene-based wireless links to enable energy-efficient, multi-modal chip-to-chip communication protocol to create toroidal folding based interconnection architectures for multichip systems. With cycle-accurate simulations we demonstrate that such designs can outperform state-of-the-art wireline multichip systems.

Estimation of distribution-based multiobjective design space exploration for energy and throughput-optimized MPSoCs

Modern multicore architectures comprise a large set of components and parameters that require being matched to achieve the best balance between power consumption and throughput performance for a particular application domain. The exploration of design space for finding the best power throughput trade-off is a combinatorial optimization problem with a large number of combinations, and. in general, black its solution is prohibitively difficult to be explored exhaustively. However, fortunately, evolutionary algorithms (EAs) have the potential to efficiently solve this problem with reasonable computational complexity. In this paper, we consider a multiobjective design space exploration (DSE) problem with two conflicting objectives. The first objective corresponds to power consumption minimization while the second objective relates to throughput maximization. We approach this problem by employing the estimation of distribution algorithm (EDA), which belongs to the family of EAs. The proposed EDA-based DSE (EDA-DSE) scheme efficiently selects the design parameters (i.e. cache size, number of cores, and operating frequency) with an efficient power through put ratio. The proposed scheme is verified using cycle accurate simulations over a set of benchmarks and the simulation results show a significant reduction in energy delay product for all benchmark applications when compared to the default baseline configuration and genetic algorithm.

A design flow for an optimized congestion-aware application-specific wireless network-on-chip architecture

Wireless Networks-on-Chip (WiNoC) architecture has emerged as an alternative communication infrastructure for the conventional wire-line NoC to achieve higher performance and low energy dissipation in on-chip communications. However, as realistic applications have different communication requirements, application-specific platforms are attractive to guarantee a specific level of performance. Moreover, the congestion probability over wireless routers is high, because each wireless router is shared by a group of cores. Therefore, we propose a novel methodology for designing an optimized congestion-aware application-specific WiNoC architecture. This methodology utilizes a novel mesh-of-tree (MoT) topology as a communication infrastructure to exploit the benefits of both mesh and tree topologies. Then, for a given application, the long-distance wire/wireless links are inserted into the MoT topology by considering the optimization of system cost and performance simultaneously. Furthermore, a congestion-aware layered routing is presented to reduce the congestion probability. Through cycle-accurate simulations, the performance of the proposed architecture has been evaluated and compared with state-of-the-art works under both synthetic and realistic traffic patterns in terms of network throughput, latency, and energy consumption. Moreover, the area overhead associated with the proposed WiNoC architecture is investigated. The experimental results confirmed that the proposed MoT-based WiNoC architecture is a very competitive architecture among the alternative WiNoC architectures.