Heterogeneous Processing Elements Research Articles

Systems-on-Chip with Strong Ordering

Sequential consistency (SC) is the most intuitive memory consistency model and the easiest for programmers and hardware designers to reason about. However, the strict memory ordering restrictions imposed by SC make it less attractive from a performance standpoint. Additionally, prior high-performance SC implementations required complex hardware structures to support speculation and recovery. In this article, we introduce the lockstep SC consistency model (LSC), a new memory model based on SC but carefully defined to accommodate the data parallel lockstep execution paradigm of GPUs. We also describe an efficient LSC implementation for an APU system-on-chip (SoC) and show that our implementation performs close to the baseline relaxed model. Evaluation of our implementation shows that the geometric mean performance cost for lockstep SC is just 0.76% for GPU execution and 6.11% for the entire APU SoC compared to a baseline with a weaker memory consistency model. Adoption of LSC in future APU and SoC designs will reduce the burden on programmers trying to write correct parallel programs, while also simplifying the implementation and verification of systems with heterogeneous processing elements and complex memory hierarchies. 1

Heterogeneous Systolic Array Architecture for Compact CNNs Hardware Accelerators

Compact convolutional neural networks have become a hot research topic. However, we find that the systolic array accelerators are extremely inefficient in dealing with compact models, especially when processing depthwise convolutional layers in the neural networks. To make systolic arrays more efficient for compact convolutional neural networks, we propose the heterogeneous systolic array (HeSA) architecture. It introduces heterogeneous processing elements that support multiple modes of dataflow, which can further exploit the reuse data chance of depthwise convolutional layers and without changing the scale or structure of the nave systolic array. By increasing the utilization rate of processing elements in the array, the HeSA improves the performance, throughput, and energy efficiency compared to the standard baseline. In addition, we design the flexible buffer structure for the communication between the computing array and external buffer. Through flexible routing, the HeSA can achieve a large-scale array design with maintaining high processing elements utilization rate and low communication costs. Based on our evaluation with typical workloads, the HeSA improves the utilization rate of the computing resource in depthwise convolutional layers by 4.5 - 11.2 and acquires 1.6 - 3.1 total performance speedup compared to the standard systolic array architecture. In the large-scale array design, the HeSA can reduce the data traffic by 40% while maintaining the same performance as the scaling-out method. By improving the on-chip data reuse chance and reducing data traffic, the HeSA saves over 20% in energy consumption. Meanwhile, the area of the HeSA is basically unchanged compared to the baseline due to its simple design.

An Efficient Organization of Configuration for CGRAs Applied to Block Ciphers

An efficient organization of configuration is proposed to decrease the size of configuration memory and the time of configuration transmission. Different block ciphers are analysed. And, computing features of algorithms are summarized. Then, the configuration is organized according to the heterogeneous Processing Elements (PEs). It is applied to a coarse-grained reconfigurable architecture (CGRA) which is implemented under TSMC 40nm CMOS technology to compare with similar work. The comparison results show that the configuration capacity decreases with the proposed organization structure. Configuration capacity of the algorithm is reduced by 90.93% to 96.91%.

Contention-aware optimal scheduling of real-time precedence-constrained task graphs on heterogeneous distributed systems

Abstract Real-time applications in today’s distributed cyber-physical control systems are often represented as Precedence-constrained Task Graphs (PTGs) and increasingly implemented on heterogeneous platforms to cater to their high performance demands. Optimal scheduling solutions for such systems can provide advantages in terms of performance, reliability, cost etc. However, existing research works dealing with the optimal scheduling of PTGs, typically assume platforms consisting of homogeneous processing elements which interact through a fully connected network of homogeneous communication channels. In this work, we propose an Integer Linear Programming based optimal solution strategy for scheduling PTGs executing on a distributed platform composed of heterogeneous processing elements and inter-connected through a set of heterogeneous shared buses. Through the real-world case study of an automotive cruise controller, we generate an optimal schedule using our proposed scheme in order to demonstrate its generic applicability. Conducted experiments on benchmark PTGs reveal the practical efficacy of our scheme.

On-Chip Hardware Accelerator for DSP Applications

High speed computing systems developed for multimedia streaming application demand high throughput and which can be achieved by designing hardware accelerators for data processing. This article presents new hardware accelerating platform comprised of heterogeneous multi core processing elements integrated on single chip FPGA. This kind of multi core platform can boost multimedia applications through parallel processing. The proposed multi core platform has been realized on FPGA and few DSP applications are executed on the processing elements of the platform to validate its performance. The performance of the proposed hardware accelerator has been compared with existing standard computing platforms frequently used for multimedia applications. The comparison shows that the proposed on-chip multi core accelerator has enhanced the execution speed of DSP applications while providing optimum throughput.

On the Design of Time-Constrained and Buffer-Optimal Self-Timed Pipelines

Pipelining is a powerful technique to achieve high performance in computing systems. However, as computing platforms become large-scale and integrate with heterogeneous processing elements (PEs) (CPUs, GPUs, field-programmable gate arrays, etc.), it is difficult to employ a global clock to achieve synchronous pipelines. Therefore, self-timed (or asynchronous) pipelines are usually adopted. Nevertheless, due to their complex running behavior, the performance modeling and systematic optimizations for self-timed pipeline (STP) systems are more complicated than those for synchronous ones. This paper employs marked graph theory to model STPs and presents algorithms to detect performance bottlenecks. Based on the proposed model, we observe that the system performance can be improved by inserting buffers. Due to the limited memory resources on the PEs, it is critical to minimize the number of buffers for STPs while satisfying the required timing constraints. In this paper, we propose integer linear programming formulations to obtain the optimal solutions and devise efficient algorithms to obtain the near-optimal solutions. Experimental results show that the proposed algorithms can achieve 53.10% improvement in the maximum performance and 54.04% reduction in the number of buffers, compared with the technique for the slack matching problem.

Random test program generation for verification and validation of the Samsung Reconfigurable Processor

Abstract This paper describes a framework to verify the functional correctness of the Samsung Reconfigurable Processor, a dual-mode very-long instruction word (VLIW) and coarse-grained reconfigurable array (CGRA) processor integrated in smartphones, cameras, printers, and SmartTVs. Reflecting the reconfigurable nature of the processor, the test generator constructs an architectural model directly from the processor’s high-level hardware description files. Test programs are specified in a custom Constraint Specification Language that allows full coverage tests as well as verification of corner cases. Operations and operands are mapped to the heterogeneous processing elements of the CGRA using a guided place-and-routing algorithm. Requiring no explicit knowledge about the semantics of operations, the presented framework seamlessly supports custom ISA extensions. Experiments demonstrate that the framework is versatile, efficient, and quickly achieves a high coverage. In addition to detecting all randomly inserted faults, the generated test programs also exposed two yet unknown actual faults in the architecture.

Verification of coarse-grained reconfigurable arrays through random test programs

We propose and evaluate a framework to test the functional correctness of coarse-grained reconfigurable array (CGRA) processors for pre-silicon verification and post-silicon validation. To reflect the reconfigurable nature of CGRAs, an architectural model of the system under test is built directly from the hardware description files. A guided place-and-routing algorithm is used to map operations and operands onto the heterogeneous processing elements (PE). Test coverage is maximized by favoring unexercised parts of the architecture. Requiring no explicit knowledge about the semantics of operations, the random test program generator (RTPG) framework seamlessly supports custom ISA extensions. The proposed framework is applied to the Samsung Reconfigurable Processor, a modulo-scheduled CGRA integrated in smartphones, cameras, printers, and smart TVs. Experiments demonstrate that the RTPG is versatile, efficient, and quickly achieves a high coverage. In addition to detecting all randomly inserted faults, the generated test programs also exposed two yet unknown actual faults in the architecture.

Review of recent trends in Coarse Grain Reconfigurable Architectures for signal processing applications

Coarse grained reconfigurable architecture got the attention of researchers working in designing computing architectures for processing massive streaming data associated with the multimedia applications in portable entertainment and communication electronics. The algorithms for processing audio, video, and graphics are very complex in nature. These data intensive computation algorithms belong to the domain of signal processing. As the complexity of algorithms increases, a matching improvement in speed performance of the hardware becomes essential to maintain the quality of service. The observed growth of algorithmic complexity is much higher than the growth rate of integration density governed by Moore’s law. Also, the constraints on memory bandwidth in the traditional von Neumann architectures along with the slow growth in the battery capacity demands a paradigm shift in computer architecture design. Reconfigurable hardware architecture is proposed as a possible alternative in this regard. The reconfigurable architectures are designed to exploit the regular and repetitive structure of signal processing algorithms and the coarse grained processing elements are designed to match with the word level granularity of these complex algorithms. The research shows that the coarse grain reconfigurable architectures with heterogeneous processing elements are a better option for system design in DSP applications which exploit granularity matching between the algorithms and the processing hardware, and also the inherent parallelism of DSP algorithms for the realization of low power DSP systems.

StreamDrive: a Dynamic Dataflow Framework for Clustered Embedded Architectures

In this paper, we present StreamDrive, a dynamic dataflow framework for programming clustered embedded multicore architectures. StreamDrive simplifies development of dynamic dataflow applications starting from sequential reference C code and allows seamless handling of heterogeneous and application-specific processing elements by applications. We address issues of efficient implementation of the dynamic dataflow runtime system in the context of constrained embedded environments, which have not been sufficiently addressed by previous research. We conducted a detailed performance evaluation of the StreamDrive implementation on our Application Specific MultiProcessor (ASMP) cluster using the Oriented FAST and Rotated BRIEF (ORB) algorithm typical of image processing domain. We have used the proposed incremental development flow for the transformation of the ORB original reference C code into an optimized dynamic dataflow implementation. Our implementation has less than 10% parallelization overhead, near-linear speedup when the number of processors increases from 1 to 8, and achieves the performance of 15 VGA frames per second with a small cluster configuration of 4 processing elements and 64KB of shared memory, and of 30 VGA frames per second with 8 processors and 128KB of shared memory.

Reproducible Evaluation of System Efficiency With a Model of Architecture: From Theory to Practice

Current trends in high performance and embedded computing include design of increasingly complex hardware architectures with high parallelism, heterogeneous processing elements, and nonuniform communication resources. In order to take hardware and software design decisions, early evaluations of the system nonfunctional properties are needed. These evaluations of system efficiency require electronic system-level information on both algorithms and architecture. Contrary to algorithm models for which a major body of work has been conducted on defining formal models of computation (MoCs), architecture models from the literature are mostly empirical models from which reproducible experimentation requires the accompanying software. In this paper, a precise definition of a model of architecture (MoA) is proposed that focuses on reproducibility and abstraction and removes the overlap previously existing between the notions of MoA and MoC. A first MoA, called the linear system-level architecture model (LSLA), is presented. To demonstrate the generic nature of the proposed new architecture modeling concepts, we show that the LSLA model can be integrated flexibly with different MoCs. LSLA is then used to model the energy consumption of a state-of-the-art multiprocessor system-on-chip (MPSoC) when running an application described using the synchronous dataflow MoC. A method to automatically learn LSLA model parameters from platform measurements is introduced. Despite the high complexity of the underlying hardware and software, a simple LSLA model is demonstrated to estimate the energy consumption of the MPSoC with a fidelity of 86%.

Bio-inspired fault tolerant network on chip

The integration of heterogeneous processing elements (PEs) or nodes in the System on Chip (SoC) has made the communication structure very complex. The bus based system between these components is not able to handle the communication requirements and, this has led to the idea of Network on Chip (NoC). The NoC addresses the communication requirement of different nodes on SoC. The physical sizes of devices in NoC are scaled down, including routers, processing elements and interconnects, giving rise to faults, system delay, and latency issues. Fault tolerant routing algorithms are used to recover from temporary faults while redundant resources (wires, routers) are required to overcome the permanent faults. These routing algorithms, however, still suffer from congestion problems, low bandwidth, and throughput utilization as well as lacking adaptivity and robustness. In this work, novel biologically inspired techniques were proposed for NoC using combined best effort (BE) and guaranteed throughput (GT) services. Moreover, the bio-inspired algorithms are compared and analyzed with each other using BE, GT and combined BE-GT services. The bio-inspired mechanisms of “synaptogenesis” and “sprouting” have been adopted in the proposed NoC algorithms and architecture. These techniques were implemented using the BE and GT services. With the help of these two bio-inspired techniques, the NoC becomes robust, fault tolerant and is able to efficiently utilize the throughput and bandwidth. The bio-inspired algorithms improved the accepted traffic (flit/cycle/node) by 38.99% compared to different techniques in the literature. The bio-inspired algorithm also improved the bandwidth and throughput utilization by 71.04% and 72.42% respectively compared to the XY and Odd-Even fault tolerant routing algorithms. Moreover, the bio-inspired algorithm had less end-to-end latency and interflit arrival time by 196.44% and 88.10% respectively compared to the literature techniques of XY and Odd-Even.

A hierarchical run-time adaptive resource allocation framework for large-scale MPSoC systems

In the embedded computer system domain, MPSoC systems have become increasingly popular due to the ever-increasing performance demands of modern embedded applications. The number of processing elements in these MPSoCs also steadily increases. Whereas current MPSoCs still contain a limited number of processing elements, future MPSoCs will feature tens up to hundreds of (heterogeneous) processing elements that are all integrated on a single chip. On these future large-scale MPSoC systems, the mapping of applications onto the hardware resources plays an important role to fully explore the parallelism of applications. In this article, a hierarchical run-time adaptive resource allocation framework which uses an intelligent task remapping approach is proposed to improve the system performance for large-scale MPSoCs.

Efficient Process Network Implementation of Ray-Tracing Application on Heterogeneous Multi-Core Systems

As more mobile devices are equipped with multi-core CPUs and are required to execute many compute-intensive multimedia applications, it is important to optimize the systems, considering the underlying parallel hardware architecture. In this paper, we implement and optimize ray-tracing application tailored to a given mobile computing platform with multiple heterogeneous processing elements. In this paper, a lightweight ray-tracing application is specified and implemented in Kahn process network (KPN) model-of-computation, which is known to be suitable for the description of real-time applications. We take an open-source C/C++ implementation of ray-tracing and adapt it to KPN description in the Distributed Application Layer framework. Then, several possible configurations are evaluated in the target mobile computing platform (Exynos 5422), where eight heterogeneous ARM cores are integrated. We derive the optimal degree of parallelism and a suitable distribution of the replicated tasks tailored to the target architecture.

SORT 2014

SORT 2014

Programmer Productivity in a World of Mushy Interfaces

Since 1964, we had the notion that the instruction set architecture (ISA) is a useful and fairly opaque abstraction layer between hardware and software. Software rode hardware's performance wave while remaining gloriously oblivious to hardware's growing complexity. Unfortunately, the jig is up. We still have ISAs, but the abstraction no longer offers seamless portability---parallel software needs to be tuned for different core counts, and heterogeneous processing elements (CPUs, GPUs, accelerators) further complicate programmability. We are better at building large-scale heterogeneous processors than we are at programming them. Maintaining software across multiple current platforms is difficult and porting to future platforms is also difficult. There have been many technical responses: virtual ISAs (e.g., NVIDIA's PTX), higher-level programming interfaces (e.g., CUDA or OpenCL), and late-stage compilation and platform-specific tailoring (e.g., Android ART), etc. A team of opinionated experts, drawn from the three ASPLOS communities will examine the problem of programmer productivity in the post-ISA world, first from the perspective of their area of expertise and then noting the contributions from the other two communities. What research will save us and how? This wide-ranging debate will frame important research areas for future work while being grounded in frank discussion about what has succeeded in the past. Attendees can expect actionable insight into important research issues as well an entertaining discussion.

Programmer Productivity in a World of Mushy Interfaces

Since 1964, we had the notion that the instruction set architecture (ISA) is a useful and fairly opaque abstraction layer between hardware and software. Software rode hardware's performance wave while remaining gloriously oblivious to hardware's growing complexity. Unfortunately, the jig is up. We still have ISAs, but the abstraction no longer offers seamless portability---parallel software needs to be tuned for different core counts, and heterogeneous processing elements (CPUs, GPUs, accelerators) further complicate programmability. We are better at building large-scale heterogeneous processors than we are at programming them. Maintaining software across multiple current platforms is difficult and porting to future platforms is also difficult. There have been many technical responses: virtual ISAs (e.g., NVIDIA's PTX), higher-level programming interfaces (e.g., CUDA or OpenCL), and late-stage compilation and platform-specific tailoring (e.g., Android ART), etc. A team of opinionated experts, drawn from the three ASPLOS communities will examine the problem of programmer productivity in the post-ISA world, first from the perspective of their area of expertise and then noting the contributions from the other two communities. What research will save us and how? This wide-ranging debate will frame important research areas for future work while being grounded in frank discussion about what has succeeded in the past. Attendees can expect actionable insight into important research issues as well an entertaining discussion.

Implementation and Analysis of Biological Synaptogenesis Technique on Nodes and Interconnects for NoC Fault Tolerance

Bio-inspired Network on Chip (NoC) fault tolerant techniques are a novel way of solving the complex faulty situation in NoC. The excessive and parallel communication requirements of heterogeneous processing elements (PE’s) in NoC have made the communication structure very complex. The size of the devices are scaled down to support the complexity but the size of interconnects remains the same. Due to this interconnects have contributed to faults. Different fault tolerant techniques have been proposed. But all these conventional algorithms have drawbacks of adaptiveness and robustness. The proposed synaptogensis based bio-inspired technique is based on one of the characteristics of biological brain. This technique is robust as it makes the NoC fault-tolerant and able to reconfigure upon detection of router or interconnect faults. In this paper, two techniques based on synaptogensis algorithm have been critically analyzed. In improved algorithm, the packet network latency was reduced to 34.62%, bandwidth was efficiently utilized by 5.03% and throughput was increased by 36.36%. The bio-inspired algorithm has better accepted traffic rate as compared to the traditional fault tolerant technique.

Two‐level extensions of an artificial hormone system

SummaryThe ARTIFICIAL HORMONE SYSTEM (AHS) is a decentralized software that is able to allocate tasks in a system of heterogeneous processing elements (PEs). Tasks are allocated according to their suitability for the heterogeneous PEs, the current PE load, and task relationships. The AHS also provides properties like self‐configuration, self‐optimization, and self‐healing in the context of task allocation. In addition, it is able to guarantee real‐time bounds for such self‐X properties. However, using self‐organization principles introduces increased system complexity such as control of system parameters for self‐organization and additional communication effort. In this contribution, we address these problems by using two different two‐level extensions of the AHS: we consider the choice and control of hormone parameters by an observer/controller architecture as first extension of the AHS and a HIERARCHICAL AHS (HAHS) to save communication bandwidth as second extension of the AHS. For the first extension, we use a machine learning approach for gradually learning the hormone values of different tasks. This is a major advance because expert knowledge is needed to configure the AHS up to now. We present an observer/controller architecture as an extension of the AHS to monitor and control its behavior. The user has to provide a simple set of initial rules, and the observer/controller is able to generate new rules if needed. The evaluation of our approach using a benchmark (containing six different types of tasks) shows that the observer/controller is able to match the goals provided by the user, and we discuss it in detail. The second extension is the hierarchical AHS where the PEs of the system consist of several different clusters each of them having its own communication infrastructure, for example, a bus system. The HAHS is able to save communication effort because the broadcast communication of the hormones is limited to the clusters. Nevertheless, it provides the same self‐X properties as the AHS, even the time for self‐configuration is shorter than for the AHS. We present evaluations that show that the HAHS performs better for applications with large numbers of PEs than the AHS. Copyright © 2015 John Wiley & Sons, Ltd.

A methodology for performance/energy consumption characterization and modeling of video decoding on heterogeneous SoC and its applications

To meet the increasing complexity of mobile multimedia applications, SoCs equipping modern mobile devices integrate powerful heterogeneous processing elements among which Digital Signal Processors (DSP) and General Purpose Processors (GPP) are the most common ones. Due to the ever-growing gap between battery lifetime and hardware/software complexity in addition to application's computing power needs, the energy saving issue becomes crucial in the design of such architectures. In this context, we propose in this paper an end-to-end study of video decoding on both GPP and DSP. The study was achieved thanks to a two steps methodology: (1) a comprehensive characterization and evaluation of the performance and the energy consumption of video decoding, (2) an accurate high level energy model is extracted based on the characterization step.The characterization of the video decoding is based on an experimental methodology and was achieved on an embedded platform containing a GPP and a DSP. This step highlighted the importance of considering the end-to-end decoding flow when evaluating the energy efficiency of video decoding application. The measurements obtained in this step were used to build a comprehensive analytical energy model for video decoding on both GPP and DSP. Thanks to a sub-model decomposition, the developed model estimates the energy consumption in terms of processor clock frequency and video bit-rate in addition to a set of constant coefficients which are related to the video complexity, the operating system and the considered hardware architecture. The obtained model gave very accurate results (R2=97%) for both GPP and DSP energy consumption. Finally, based on the results emerged from the modeling methodology, we show how one can build rapidly a video decoding energy model for a given target architecture without executing the full characterization steps described in this paper.