Multiprocessor System-on-chip Research Articles

CAMDNN: Content-Aware Mapping of a Network of Deep Neural Networks on Edge MPSoCs

Machine Learning (ML) workloads are increasingly deployed at the edge. Enabling efficient inference execution while considering model and system heterogeneity remains challenging, especially for ML tasks built with a network of DNNs. The challenge is to maximize the utilization of all available resources on the multiprocessor system on a chip (MPSoC) at the same time. This becomes even more complicated because the optimal mapping for the network of DNNs can vary with input batch sizes and scene complexity. In this paper, a holistic hierarchical scheduling framework is presented to optimize the execution time for a network of DNN models on an edge MPSoC at runtime, considering varying input characteristics. The framework consists of a local and a global scheduler. The local scheduler maps individual DNNs in the inference pipeline to the best-performing hardware unit while the global scheduler customizes an Integer Linear Programming (ILP) solution to instantiate DNN remapping. To minimize scheduler runtime overhead, an imitation learning (IL) based scheduler is used that approximates the ILP solutions. The proposed scheduling framework (CAMDNN) was implemented on a Qualcomm Robotic RB5 platform. CAMDNN resulted in lower execution time of up to 32% than HEFT, and by factors of 6.67X, 5.6X and 2.17X than the CPU-only, GPU-only and Central Queue schedulers.

Discrete Firefly Algorithm for Optimizing Topology Generation and Core Mapping of Network-on-Chip

Network-on-chip (NoC) proves to be the best alternative to replace the traditional bus-based interconnection in Multi-Processor System on a Chip (MPSoCs). Irregular NoC topologies are highly recommended and utilised in various applications as they are application specific. Optimized mapping of the cores in a NoC plays a major role in its performance. Firefly algorithm is a bio-inspired meta-heuristic approach. Discretized firefly algorithm is used in our proposed work. In this work, two optimization algorithms are proposed: Topology Generation using Discrete Firefly Algorithm (TGDFA) and Core Mapping using Discrete Firefly Algorithm (CMDFA) for multimedia benchmark applications, Video Object Plane Decoder (VOPD), Multimedia Window Display (MWD) and MP3 Encoder. The irregular topology generated using TGDFA is mapped using CMDFA onto a reconfigurable mesh with switches in between the routers to make it a fault tolerant one. The proposed TGDFA provides better optimization of communication cost. The speed-up on the average run time of the tasks and the time taken to attain the best solution by the proposed TGDFA is significant. The dynamic energy consumption of core mapping obtained by the proposed CMDFA is lesser when compared to the existing work. Finally, the optimized core mapping is implemented in a cycle accurate NoC simulator: Noxim. It is substantiated experimentally that the proposed CMDFA outperforms the previous work.

Review, Analysis, and Implementation of Path Selection Strategies for 2D NoCs

Recent advances in very-large-scale integration (VLSI) technologies have offered the capability of integrating thousands of processing elements onto a single silicon microchip. Multiprocessor systems-on-chips (MPSoCs) are the latest creation of this technology evolution. Network-on-Chip (NoC) is a scalable and promising interconnection solution used by MPSoCs to achieve high performance. Routing algorithms provide a path to a packet toward the destination. For this, these algorithms should exhibit two characteristics. First, the route selection function should provide enough degree of adaptiveness to avoid network congestion. Second, it should not offer stale information on network congestion status to the neighboring routers. Many researchers have investigated network congestion and proposed techniques to control/avoid congestion. Such congestion avoidance-based algorithms significantly improve NoC performance. However, they may result in hardware overhead for side network implementation to collect congestion status. This paper reviews the selection strategies used to reduce congestion in NoC and classifies them on the technique adopted to handle and propagate congestion information. Additionally, this paper provides the implementation and analysis details of some state-of-art selection methods.

Reinforcement Learning Based Fault-Tolerant Routing Algorithm for Mesh Based NoC and Its FPGA Implementation

Network-on-Chip (NoC) has emerged as the most promising on-chip interconnection framework in Multi-Processor System-on-Chips (MPSoCs) due to its efficiency and scalability. In the deep sub-micron level, NoCs are vulnerable to faults, which leads to the failure of network components such as links and routers. Failures in NoC components diminish system efficiency and reliability. This paper proposes a Reinforcement Learning based Fault-Tolerant Routing (RL-FTR) algorithm to tackle the routing issues caused by link and router faults in the mesh-based NoC architecture. The efficiency of the proposed RL-FTR algorithm is examined using System-C based cycle-accurate NoC simulator. Simulations are carried out by increasing the number of links and router faults in various sizes of mesh. Followed by simulations, real-time functioning of the proposed RL-FTR algorithm is observed using the FPGA implementation. Results of the simulation and hardware shows that the proposed RL-FTR algorithm provides an optimal routing path from the source router to the destination router.

PathTracer: Understanding Response Time of Signal Processing Applications on Heterogeneous MPSoCs

In embedded and cyber-physical systems, the design of a desired functionality under constraints increasingly requires parallel execution of a set of tasks on a heterogeneous architecture. The nature of such parallel systems complicates the process of understanding and predicting performance in terms of response time. Indeed, response time depends on many factors related to both the functionality and the target architecture. State-of-the-art strategies derive response time by examining the operations required by each task for both processing and accessing shared resources. This procedure is often followed by the addition or elimination of potential interference due to task concurrency. However, such approaches require an advanced knowledge of the software and hardware details, rarely available in practice. This work presents an alternative “top-down” strategy, called PathTracer, aimed at understanding software response time and extending the cases in which it can be analyzed and estimated. PathTracer leverages on dataflow-based application representation and response time estimation of signal processing applications mapped on heterogeneous Multiprocessor Systems-on-a-Chip (MPSoCs). Experimental results demonstrate that PathTracer provides (i) information on the nature of the application (work-dominated, span-dominated, or balanced parallel), and (ii) response time modeling which can reach high accuracy when performed post-execution, leading to prediction errors with average and standard deviation under 5% and 3% respectively.

A Review of Design Approaches for Enhancing the Performance of NoCs at Communication Centric Level

As the trend of technology shrinking continues a vast amount of processors are being incorporated in a limited space. Due to this almost half of the chip area in Multi-Processor Systems-on-Chips (MPSoCs) is under interconnections, which pose a big problem for communication. Network-on-Chips (NoCs) evolved as a significant scalable solution for removing wiring congestion and communication problem in MPSoCs. NoCs provide the advantage of customized architecture, increased scalability and bandwidth. NoC is a structured framework where communication is the prime concern. In this review paper we present an overview of research and design approaches in the communication centric areas of NoCs. Here we have tried to discuss and iterate most of the available work done for communication in 2D NoCs. This paper gives the insight of different attributes and performance parameters of NoCs. Further it gives a detailed description of how topology, flow control and routing mechanisms can affect the qualitative aspects (performance) of NoCs. It then explains how various attributes of routing can help in increasing the efficacy of NoCs. Subsequently a brief review of different simulators used for NoCs is given. All of this is provided based on the survey of academic, theoretical and experimental approaches presented in the past. Finally some suggestions for future work are also given.

Reinforcement Learning-Based Power Management Policy for Mobile Device Systems

This paper presents a power management policy that utilizes reinforcement learning to increase the power efficiency of mobile device systems based on a multiprocessor system-on-a-chip (MPSoC). The proposed policy predicts a system’s characteristics and learns power management controls to adapt to the variations in the system. We consider the behavioral characteristics of systems that run on mobile devices under diverse scenarios. Therefore, the policy can flexibly manage the system power regardless of the application scenario and achieve lower energy consumption without compromising the user satisfaction. The average energy per unit quality of service (QoS) of the proposed policy is lower than that of the previous six dynamic voltage/frequency scaling governors by 31.66%. Furthermore, we reduce the runtime overhead by implementing the proposed policy as hardware. We implemented the policy on the field programmable gate array (FPGA) and construct a communication interface between the central processing units (CPUs) and the hardware of the proposed policy. Decision-making by the hardware-implemented policy is 3.92 times faster than by the software-implemented policy.

An energy efficient synthesis flow for application specific SoC design

Multiple Supply Voltage (MSV) is a known design technique to deal with the power and thermal issues of Multi Processor System on Chips (MPSoCs). In this paper, an MSV driven synthesis flow is proposed in four phases for the design of application-specific MPSoCs. The first phase conducts the core-to-router allocation concerning the operating voltages of cores as well as the bandwidth/latency requirements of the data flowing between the cores. The second phase coordinates voltages of cores connected to the same router. The third phase refines the power delivery network using our proposed voltage-aware hierarchical floorplanning algorithm. Finally, router-to-router connection and path allocation are done by the use of existing methods in the last phase. For the first time, we perform the core to router allocation phase before the voltage islanding phase to find the highest number of design solutions satisfying the set of given constraints; this in turn, improves the flexibility and efficiency of our synthesis process. Exhaustive experimental evaluations have been done on real-world benchmarks. Results confirm achieving 90% more design solutions (to conduct a wiser design space exploration) and an average of 93% Energy Delay Product (EDP) improvement as compared to the existing synthesis approaches.

The Performance Analysis of Thermal Effect in Mesh-Based Optical Networks-on-Chips

Optical networks-on-chips (ONoCs) are the key technology for the sustainable development of multiprocessors system-on-chip (MPSoC) in the future. Micro-ring resonators (MRs) are widely used in ONoCs as key device to select and redirect optical signals. However, MRs have the inherent property of being sensitive to environmental temperature. With the fluctuation of environmental temperature, its resonance wavelength drifts, which can introduce more loss and crosstalk noise to ONoCs and make the network performance decline sharply. Therefore, it is very important to analyze the influence of thermal effect in ONoCs to solve this problem. In this paper, the theoretical models of loss and crosstalk noise changing with temperature are established from device level to network level, respectively. And a series of network performances of mesh-based ONoCs caused by thermal effect are systematically modeled and analyzed employing formal methods. Finally, we conduct case studies for mesh-based ONoCs using optimized crossbar optical router and crux optical router to evaluate the proposed method. The simulation results show that the performance of mesh-based ONoCs declines with the increase of temperature, such as the decrease of optical signal-to-noise ratio (OSNR) and the increase of bit error rate (BER), which severely limits the network scalability. The formal analytical models provide a criterion for the performance analysis of thermal effect in ONoCs. In addition, the formal methods have high portability and scalability, and can provide technical support for future research work.

Expected Energy Optimization for Real-Time Multiprocessor SoCs Running Periodic Tasks with Uncertain Execution Time

Energy optimization plays an increasingly critical role in designing an embedded real-time multiprocessor System on Chip (MPSoC). Dynamic Voltage Frequency Scaling (DVFS) and Dynamic Power Management (DPM) are preferable techniques to optimize energy consumption. However, previous DVFS and DPM algorithms were mostly designed for inter-task scheduling, without sufficient exploration on intra-task scheduling for further energy reduction. This paper presents a new intra-task scheduling approach considering the probabilistic distribution of task execution time, and it optimizes the mathematical expectation of power consumption (expected power consumption) for periodic dependent tasks with uncertain execution time running on MPSoCs using DVFS and DPM. The energy-efficient scheduling problem can be formulated by means of mixed integer linear programming (MILP) with the proposed technique. Moreover, we also propose a technique to compress the exploration space by reorganizing the probabilistic profiling information of all tasks. Our experimental results on synthetic and realistic benchmarks show that the proposed approach achieves up to 30 percent energy savings compared with other existing methods.

Argus CNN Accelerator Based on Kernel Clustering and Resource-Aware Pruning

Paper proposes a two-step Convolutional Neural Network (CNN) pruning algorithm and resource-efficient Field-programmable gate array (FPGA) CNN accelerator named “Argus”. The proposed CNN pruning algorithm first combines similar kernels into clusters, which are then pruned using the same regular pruning pattern. The pruning algorithm is carefully tailored for FPGAs, considering their resource characteristics. Regular sparsity results in high Multiply-accumulate (MAC) efficiency, reducing the amount of logic required to balance workloads among different MAC units. As a result, the Argus accelerator requires about 170 Look-up tables (LUTs) per Digital Signal Processor (DSP) block. This number is close to the average LUT/DPS ratio for various FPGA families, enabling balanced resource utilization when implementing Argus. Benchmarks conducted using Xilinx Zynq Ultrascale + Multi-Processor System-on-Chip (MPSoC) indicate that Argus is achieving up to 25 times higher frames per second than NullHop, 2 and 2.5 times higher than NEURAghe and Snowflake, respectively, and 2 times higher than NVDLA. Argus shows comparable performance to MIT’s Eyeriss v2 and Caffeine, requiring up to 3 times less memory bandwidth and utilizing 4 times fewer DSP blocks, respectively. Besides the absolute performance, Argus has at least 1.3 and 2 times better GOP/s/DSP and GOP/s/Block-RAM (BRAM) ratios, while being competitive in terms of GOP/s/LUT, compared to some of the state-of-the-art solutions.

ThermalAttackNet: Are CNNs Making It Easy to Perform Temperature Side-Channel Attack in Mobile Edge Devices?

Side-channel attacks remain a challenge to information flow control and security in mobile edge devices till this date. One such important security flaw could be exploited through temperature side-channel attacks, where heat dissipation and propagation from the processing cores are observed over time in order to deduce security flaws. In this paper, we study how computer vision-based convolutional neural networks (CNNs) could be used to exploit temperature (thermal) side-channel attack on different Linux governors in mobile edge device utilizing multi-processor system-on-chip (MPSoC). We also designed a power- and memory-efficient CNN model that is capable of performing thermal side-channel attack on the MPSoC and can be used by industry practitioners and academics as a benchmark to design methodologies to secure against such an attack in MPSoC.

An Adaptive Fault-Tolerant and Congestion Controlled Selection Strategy for Networks on Chip

Background & Objective: Networks on chip are being developed as a communication infrastructure in the design of multiprocessor SOCs. With the reduction in feature size, transient faults on the links are becoming a major issue on the performance of NOCs. Methods: In this paper, two fault-tolerant algorithms are proposed. In the first algorithm, a faulty link tolerant algorithm is designed which by measuring network loads on the links will reduce transient faults and balances the load. To address the effect of hardware faults, fault and congestion controlled algorithm is designed that not only control the congestion, but also the faults on both links and the nodes. Results: The packet size is taken as 8 flits and data width of 32 bits are considered for all switches. Input buffers are having 8 slots for storing data. The simulations are carried over two synthetic traffic scenarios that are, hotspot and transpose. Conclusion: The proposed strategies are evaluated on two different synthetic traffic patterns and the results so obtained shows better network and hardware performance of both the routing in comparison with non-fault-tolerant routing.

A Real-Time Low-Power Coding Bit-Rate Control Scheme for High-Efficiency Video Coding in a Multiprocessor System-on-Chip

A real-time high-performance transmission bandwidth-aware (TB-aware) coding bit-rate (CBR) controller design with low power consumption and low hardware complexity is presented in this article for H.265/high-efficiency video coding (HEVC) in a multiprocessor system-on-chip (MPSoC). Previous TB-aware motion estimation designs with CBR-control capability in video coding have focused on algorithm development with precise CBR models, which require a complicated algorithmic derivation according to the system on-demand CBR and are difficult to realize in very large scale integration (VLSI) due to their lack of consideration for hardware implementation and modeling. Consequently, we present a hardware-oriented CBR-control algorithm that uses simple CBR control functions instead of requiring root and exponential operations to realize the real-time low-power design objective for HEVC applications within a mobile MPSoC. Then, an adequate hardware architecture with low hardware complexity is exploited to accomplish a low-power and high-speed VLSI design of a CBR controller for our proposed algorithm. Using diverse video-sized sequences under on-demand system coding-bit-rate constraints, the experimental outcomes demonstrate that the introduced design is capable of low power consumption and high speeds and can utilize low-complexity hardware.

3D-ICE 3.0: Efficient Nonlinear MPSoC Thermal Simulation With Pluggable Heat Sink Models

The increasing power density in modern high-performance multiprocessor System-on-Chip (MPSoC) is fueling a revolution in thermal management. On the one hand, thermal phenomena are becoming a critical concern, making accurate and efficient simulation a necessity. On the other hand, a variety of physically heterogeneous solutions is coming into play: liquid, evaporative, thermoelectric cooling, and more. A new generation of simulators, with unprecedented flexibility, is thus required. In this article, we present 3D-ICE 3.0, the first thermal simulator to allow for accurate nonlinear descriptions of complex and physically heterogeneous heat dissipation systems, while preserving the efficiency of latest compact modeling frameworks at the silicon die level. 3D-ICE 3.0 allows designers to extend the thermal simulator with new heat sink models while simplifying the time-consuming step of model validation. The support for nonlinear dynamic models is included, for instance, to accurately represent variable coolant flows. Our results present validated models of a commercial water heat sink and an air heat sink plus fan that achieve an average error below 1 °C and simulate, respectively, up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3\times $ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$12\times $ </tex-math></inline-formula> faster than the real physical phenomena.

Enabling Parallelized-QEMU for Hardware/Software Co-Simulation Virtual Platforms

Presently, the trend is to increase the number of cores per chip. This growth is appreciated in Multi-Processor System-On-Chips (MPSoC), composed of more cores in heterogeneous and homogeneous architectures in recent years. Thus, the difficulty of verification of this type of system has been great. The hardware/software co-simulation Virtual Platforms (VP) are presented as a perfect solution to address this complexity, allowing verification by simulation/emulation of software and hardware in the same environment. Some works parallelized the software emulator to reduce the verification times. An example of this parallelization is the QEMU (Quick EMUlator) tool. However, there is no solution to synchronize QEMU with the hardware simulator in this new parallel mode. This work analyzes the current software emulators and presents a new method to allow an external synchronization of QEMU in its parallelized mode. Timing details of the cores are taken into account. In addition, performance analysis of the software emulator with the new synchronization mechanism is presented, using: (1) a boot Linux for MPSoC Zynq-7000 (dual-core ARM Cortex-A9) (Xilinx, San Jose, CA, USA); (2) an FPGA-Linux co-simulation of a power grid monitoring system that is subsequently implemented in an industrial application. The results show that the novel synchronization mechanism does not add any appreciable computational load and enables parallelized-QEMU in hardware/software co-simulation virtual platforms.

Temperature-Constrained Reliability Optimization of Industrial Cyber-Physical Systems Using Machine Learning and Feedback Control

As the backbone of Industry 4.0, industrial cyber-physical systems (ICPSs) that are geographically dispersed, federated, cooperative, and security-critical systems become the center of interest from both industry and academia. In ICPS, there are huge amounts of devices, such as sensors and actuators, which are embedded and networked together to improve the performance of real-time monitoring and control. Reliability and temperature are two important concerns of these embedded and networked devices in ICPS due to their stringent requirement of reliable execution and long lifespan. In this article, we study the problem of maximizing soft-error reliability of CPU- and GPU-integrated embedded platforms deployed in ICPS under the temperature constraint. To speed up the estimation of soft-error rate (SER) and temperature, we train an artificial neural network (ANN) that is able to quickly and accurately derive the system’s SER and temperature. To solve the temperature-constrained reliability optimization problem, we propose a feedback control-based task scheduling scheme that adaptively determines the number of tasks admitted in the system and the number of replicas for the admitted tasks. We perform a series of simulation experiments to verify the efficacy of our scheme. The experimental results demonstrate that: 1) the estimated SER and temperature derived by our ANN-based method are very close to the ground-truth data and 2) our proposed feedback control-based task scheduling method can improve system reliability by up to 184.2% with a lower peak temperature when compared with one baseline and two state-of-the-art methods. Note to Practitioners—This article is motivated by the safety-critical industrial cyber-physical system (ICPS) applications necessitating reliable execution and long lifespan, which could be realized by increasing reliability and controlling operating temperature. Our goal is to improve the system reliability of CPU- and GPU-integrated multiprocessor systems-on-chip (MPSoCs) deployed in ICPS under the temperature constraint. Most of the existing papers target either reliability or temperature. A few recent papers have focused on reliability and temperature optimization simultaneously. However, they are not designed for ICPS and do not consider the widely accepted CPU- and GPU-integrated MPSoC platforms. This article proposes a machine learning-based approach that trains an artificial neural network (ANN) to facilitate the online estimation of system SER and temperature. Compared to the offline estimation using simulation tools, the online approach is more applicable to real-time ICPS applications. This article also designs a feedback control-based approach for improving system reliability and reducing peak temperature of the CPU- and GPU-integrated MPSoCs by determining the number of tasks to be admitted and the number of replicas for tasks.

A Non-Intrusive Tool Chain to Optimize MPSoC End-to-End Systems

Multi-core systems are now found in many electronic devices. But does current software design fully leverage their capabilities? The complexity of the hardware and software stacks in these platforms requires software optimization with end-to-end knowledge of the system. To optimize software performance, we must have accurate information about system behavior and time losses. Standard monitoring engines impose tradeoffs on profiling tools, making it impossible to reconcile all the expected requirements: accurate hardware views, fine-grain measurements, speed, and so on. Subsequently, new approaches have to be examined. In this article, we propose a non-intrusive, accurate tool chain, which can reveal and quantify slowdowns in low-level software mechanisms. Based on emulation, this tool chain extracts behavioral information (time, contention) through hardware side channels, without distorting the software execution flow. This tool consists of two parts. (1) An online acquisition part that dumps hardware platform signals. (2) An offline processing part that consolidates meaningful behavioral information from the dumped data. Using our tool chain, we studied and propose optimizations to MultiProcessor System on Chip (MPSoC) support in the Linux kernel, saving about 60% of the time required for the release phase of the GNU OpenMP synchronization barrier when running on a 64-core MPSoC.

Energy-Aware Scheduling of Streaming Applications on Edge-Devices in IoT-Based Healthcare

The reliance on Network-on-Chip (NoC)-based Multiprocessor Systems-on-Chips (MPSoCs) is proliferating in modern embedded systems to satisfy the higher performance requirement of multimedia streaming applications. Task level coarse grained software pipeling also called re-timing when combined with Dynamic Voltage and Frequency Scaling (DVFS) has shown to be an effective approach in significantly reducing energy consumption of the multiprocessor systems at the expense of additional delay. In this article we develop a novel energy-aware scheduler considering tasks with conditional constraints on Voltage Frequency Island (VFI)-based heterogeneous NoC-MPSoCs deploying re-timing integrated with DVFS for real-time streaming applications. We propose a novel task level re-timing approach called R-CTG and integrate it with non linear programming-based scheduling and voltage scaling approach referred to as ALI-EBAD. The R-CTG approach aims to minimize the latency caused by re-timing without compromising on energy-efficiency. Compared to R-DAG, the state-of-the-art approach designed for traditional Directed Acyclic Graph (DAG)-based task graphs, R-CTG significantly reduces the re-timing latency because it only re-times tasks that free up the wasted slack. To validate our claims we performed experiments on using 12 real benchmarks, the results demonstrate that ALI-EBAD out performs CA-TMES-Search and CA-TMES-Quick task schedulers in terms of energy-efficiency.

Fault-Tolerant Application Mapping on Mesh-of-Tree based Network-on-Chip

Abstract Network-on-Chip (NoC) has been considered as an efficient communication infrastructure to support Multi-Processor System-on-Chip (MPSoC) design requirements for future generation computing. With the increased application requirements and rapid scale down in VLSI technology, the probability of forming thermal hotspots in the processing elements is relatively high which may lead to system failure. Therefore, efficient fault-tolerant techniques are required to improve the reliability of a system by addressing the failures that may occur at different component levels in NoC. This paper proposes fault-tolerant NoC design methodologies to address the core failures that may occur in an application. An Integer Linear Programming (ILP) based mathematical formulation and Particle Swarm Optimization (PSO) based evolutionary approach have been proposed to perform fault-tolerant mapping using spare cores onto the Mesh-of-Tree (MoT) network. In the event of core failures, spare cores are used to enhance system reliability. Most of the approaches have fixed the position of spare core in the networks while performing fault-tolerant application-mapping onto NoCs. In contrast to fixing the position of spare core in MoT networks, flexibility is provided by our approaches ILP and PSO to place the spare core in the network. We have experimented with multimedia applications and synthetic applications generated using TGFF tool in static and dynamic environments. In static environment, the experiments are performed (a) by scaling the MoT network size with fixed router fault-percentage, (b) by varying the router fault-percentage in the MoT network, and (c) by considering multiple failed cores in the application. In dynamic environment, the experiments are carried out using cycle-accurate NoC simulator and performance parameters namely network latency, throughput, and power consumption are analyzed. The experimental results have shown significant improvements using our approach over the approaches proposed in the literature.