Design Space Exploration Framework Research Articles

Design of Fully Spectral CNNs for Efficient FPGA-Based Acceleration.

Computing convolutional layers in the frequency domain using fast Fourier transformation (FFT) has been demonstrated to be effective in reducing the computational complexity of convolutional neural networks (CNNs). Nevertheless, the main challenge of this approach lies in the frequent and repeated transformations between the spatial and frequency domains due to the absence of nonlinear functions in the spectral domain, as such it makes the benefit less attractive for low-latency inference, especially on embedded platforms. To overcome the drawbacks in the existing FFT-based convolution, we propose a fully spectral CNN using a novel spectral-domain adaptive rectified linear unit (ReLU) layer, which completely removes the compute-intensive transformations between the spatial and frequency domains within the network. The proposed fully spectral CNNs maintain the nonlinearity of the spatial CNNs while taking into account the hardware efficiency. We then propose a deeply customized and compute-efficient hardware architecture to accelerate the fully spectral CNN inference on field programmable gate array (FPGA). Different hardware optimizations, such as spectral-domain intralayer and interlayer pipeline techniques, are introduced to further improve the performance of throughput. To achieve a load-balanced pipeline, a design space exploration (DSE) framework is proposed to optimize the resource allocation between hardware modules according to the resource constraints. On an Intel's Arria 10 SX160 FPGA, our optimized accelerator achieves a throughput of 204 Gop/s with 80% of compute efficiency. Compared with the state-of-the-art spatial and FFT-based implementations on the same device, our accelerator is 4×∼ 6.6× and 3.0×∼ 4.4× faster while maintaining a similar level of accuracy across different benchmark datasets.

MoDSE: A High-Accurate Multiobjective Design Space Exploration Framework for CPU Microarchitectures

MoDSE: A High-Accurate Multiobjective Design Space Exploration Framework for CPU Microarchitectures

DeepFlow: A Cross-Stack Pathfinding Framework for Distributed AI Systems

Over the past decade, machine learning model complexity has grown at an extraordinary rate, as has the scale of the systems training such large models. However, there is an alarmingly low hardware utilization (5–20%) in large scale AI systems. The low system utilization is a cumulative effect of minor losses across different layers of the stack, exacerbated by the disconnect between engineers designing different layers spanning across different industries. To address this challenge, in this work we designed a cross-stack performance modelling and design space exploration framework. First, we introduce CrossFlow, a novel framework that enables cross-layer analysis all the way from the technology layer to the algorithmic layer. Next, we introduce DeepFlow (built on top of CrossFlow using machine learning techniques) to automate the design space exploration and co-optimization across different layers of the stack. We have validated CrossFlow’s accuracy with distributed training on real commercial hardware and showcase several DeepFlow case studies demonstrating pitfalls of not optimizing across the technology-hardware-software stack for what is likely the most important workload driving large development investments in all aspects of computing stack.

IDeSyDe: Systematic Design Space Exploration via Design Space Identification

Design space exploration (DSE) is a key activity in embedded design processes, where a mapping between applications and platforms that meets the process design requirements must be found. Finding such mappings is very challenging due to the complexity of modern embedded platforms and applications. DSE tools aid in this challenge by potentially covering sections of the design space that could be unintuitive to designers, leading to more optimised designs. Despite this potential benefit, DSE tools remain relatively niche in the embedded industry. A significant obstacle hindering their wider adoption is integrating such tools into embedded design processes. We present two contributions that address this integration issue. First, we present the design space identification (DSI) approach for systematically constructing DSE solutions that are modular and tuneable. Modularity means that DSE solutions can be reused to construct other DSE solutions, while tuneability means that the most specific DSE solution is chosen for the target DSE problem. Moreover, DSI enables transparent cooperation between exploration algorithms. Second, we present IDeSyDe, an extensible DSE framework for DSE solutions based on DSI. IDeSyDe allows extensions to be developed in different programming languages in a manner compliant with the DSI approach. We showcase the relevance of these contributions through five different case studies. The case study evaluations showed that non-exploration DSI procedures create overheads, which are marginal compared to the exploration algorithms. Empirically, most evaluations average 2% of the total DSE request. More importantly, the case studies have shown that IDeSyDe indeed provides a modular and incremental framework for constructing DSE solutions. In particular, the last case study required minimal extensions over the previous case studies so that support for a new application type was added to IDeSyDe.

Comparison of Reinforcement Learning and Model Predictive Control for Automated Generation of Optimal Control for Dynamic Systems within a Design Space Exploration Framework

Comparison of Reinforcement Learning and Model Predictive Control for Automated Generation of Optimal Control for Dynamic Systems within a Design Space Exploration Framework

Efficient Error Estimation for High-Level Design Space Exploration of Approximate Computing Systems

This article presents an error estimation technique for a data-flow graph (DFG) representation of an approximate computing (AC) circuit. The technique, which may be used during the high-level design of digital circuits, estimates error metrics for the outputs of the approximate circuit. The proposed technique receives as an input, output error characterizations of arithmetic modules in a high-level library. The error modeling of a library module is accomplished by dividing the input (operand) ranges into intervals and then characterizing the output error for different combinations of these input intervals. Subsequently, the error for each combination is stored in a lookup table (LUT). The module error models are integrated into a design space exploration (DSE) framework to evaluate different combinations of exact and approximate realizations of various operations in the DFG. The DSE, which performs error calculation and propagation from inputs to outputs of the target DFG, may be used to explore trade-offs between the output error and other design metrics of the approximate circuit, e.g., energy efficiency. The efficacy of the proposed method is assessed for three image processing benchmarks. Results for these benchmarks demonstrate that the framework can efficiently generate the Pareto frontier (PF) in the trade-off space of accuracy versus energy efficiency for the targeted benchmarks. Compared to a purely simulation-based exploration, the proposed technique provides an average of 92 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times$</tex-math> </inline-formula> speed improvement.

Hadamard product-based in-memory computing design for floating point neural network training

Deep neural networks (DNNs) are one of the key fields of machine learning. It requires considerable computational resources for cognitive tasks. As a novel technology to perform computing inside/near memory units, in-memory computing (IMC) significantly improves computing efficiency by reducing the need for repetitive data transfer between the processing and memory units. However, prior IMC designs mainly focus on the acceleration for DNN inference. DNN training with the IMC hardware has rarely been proposed. The challenges lie in the requirement of DNN training for high precision (e.g. floating point (FP)) and various operations of tensors (e.g. inner and outer products). These challenges call for the IMC design with new features. This paper proposes a novel Hadamard product-based IMC design for FP DNN training. Our design consists of multiple compartments, which are the basic units for the matrix element-wise processing. We also develop BFloat16 post-processing circuits and fused adder trees, laying the foundation for IMC FP processing. Based on the proposed circuit scheme, we reformulate the back-propagation training algorithm for the convenience and efficiency of the IMC execution. The proposed design is implemented with commercial 28 nm technology process design kits and benchmarked with widely used neural networks. We model the influence of the circuit structural design parameters and provide an analysis framework for design space exploration. Our simulation validates that MobileNet training with the proposed IMC scheme saves in energy and in time versus the same task with NVIDIA GTX 3060 GPU. The proposed IMC design has a data density of 769.2 Kb mm−2 with the FP processing circuits included, showing a 3.5 × improvement than the prior FP IMC designs.

QUIDAM: A Framework for Qu ant i zation-aware D NN A ccelerator and M odel Co-Exploration

As the machine learning and systems communities strive to achieve higher energy efficiency through custom deep neural network (DNN) accelerators, varied precision or quantization levels, and model compression techniques, there is a need for design space exploration frameworks that incorporate quantization-aware processing elements into the accelerator design space while having accurate and fast power, performance, and area models. In this work, we present QUIDAM , a highly parameterized quantization-aware DNN accelerator and model co-exploration framework. Our framework can facilitate future research on design space exploration of DNN accelerators for various design choices such as bit precision, processing element type, scratchpad sizes of processing elements, global buffer size, number of total processing elements, and DNN configurations. Our results show that different bit precisions and processing element types lead to significant differences in terms of performance per area and energy. Specifically, our framework identifies a wide range of design points where performance per area and energy varies more than 5× and 35×, respectively. With the proposed framework, we show that lightweight processing elements achieve on par accuracy results and up to 5.7× more performance per area and energy improvement when compared to the best 16-bit integer quantization–based implementation. Finally, due to the efficiency of the pre-characterized power, performance, and area models, QUIDAM can speed up the design exploration process by three to four orders of magnitude as it removes the need for expensive synthesis and characterization of each design.

FARSI: An Early-stage Design Space Exploration Framework to Tame the Domain-specific System-on-chip Complexity

Domain-specific SoCs (DSSoCs) are an attractive solution for domains with extremely stringent power, performance, and area constraints. However, DSSoCs suffer from two fundamental complexities. On the one hand, their many specialized hardware blocks result in complex systems and thus high development effort. On the other hand, their many system knobs expand the complexity of design space, making the search for the optimal design difficult. Thus to reach prevalence, taming such complexities is necessary. To address these challenges, in this work, we identify the necessary features of an early-stage design space exploration framework that targets the complex design space of DSSoCs and provide an instance of one such framework that we refer to as FARSI. FARSI provides an agile system-level simulator with speed up and accuracy of 8,400× and 98.5% compared to Synopsys Platform Architect. FARSI also provides an efficient exploration heuristic and achieves up to 62× and 35× improvement in convergence time compared to the classic simulated annealing (SA) and modern Multi-Objective Optimistic Search. This is done by augmenting SA with architectural reasoning such as locality exploitation and bottleneck relaxation. Furthermore, we embed various co-design capabilities and show that, on average, they have a 32% impact on the convergence rate. Finally, we demonstrate that using development-cost-aware policies can lower the system complexity, both in terms of the component count and variation by as much as 60% and 82% (e.g., for Network-on-a-Chip subsystem), respectively.

AHA: An Agile Approach to the Design of Coarse-Grained Reconfigurable Accelerators and Compilers

With the slowing of Moore’s law, computer architects have turned to domain-specific hardware specialization to continue improving the performance and efficiency of computing systems. However, specialization typically entails significant modifications to the software stack to properly leverage the updated hardware. The lack of a structured approach for updating the compiler and the accelerator in tandem has impeded many attempts to systematize this procedure. We propose a new approach to enable flexible and evolvable domain-specific hardware specialization based on coarse-grained reconfigurable arrays (CGRAs). Our agile methodology employs a combination of new programming languages and formal methods to automatically generate the accelerator hardware and its compiler from a single source of truth. This enables the creation of design-space exploration frameworks that automatically generate accelerator architectures that approach the efficiencies of hand-designed accelerators, with a significantly lower design effort for both hardware and compiler generation. Our current system accelerates dense linear algebra applications but is modular and can be extended to support other domains. Our methodology has the potential to significantly improve the productivity of hardware-software engineering teams and enable quicker customization and deployment of complex accelerator-rich computing systems.

AERO: Design Space Exploration Framework for Resource-Constrained CNN Mapping on Tile-Based Accelerators

Analog In-Memory Compute (AIMC) arrays can store weights and perform matrix-vector multiplication operations for Deep Convolutional Neural Networks (CNNs). A number of recent efforts have integrated AIMC arrays into hybrid digital-analog accelerators in a multi-layer parallel manner to achieve energy efficiency and high throughput. Multi-layer parallelism on large-scale tile-based architectures need efficient mapping support at the processing element (PE)-level ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , digital or analog processing elements) and tile-level. To find the most efficient architectures, fast and accurate design space exploration (DSE) support is required. In this paper, a novel DSE framework, AERO, is presented to characterize a CNN inference workload executing on hybrid tile-based architectures that supports multi-layer parallelism. Three characteristics can be seen in our DSE framework: (1) It presents a hierarchical Tile/PE-level mapping exploration strategy including inter-layer interaction, and allowing layer fusion/splitting configurations for PE-level mapping optimization. (2) It unlocks different Performance, Power and Area (PPA) exploration points under both sufficient and limited resource constraints, while limited resource case is not considered in prior works of multi-layer parallel architectures. The impact of weight loading and weight stationary mapping are analyzed for better insights into hybrid tile-based architectures. (3) It incorporates a detailed PPA model that supports a broad range of hybrid digital and analog units in a tile. Experimental case-studies are performed for realistic and relevant benchmarks such as MLP, CNNs (Lenet-5, Resnet-18,-34,-50 and −101).

A Formalism of DNN Accelerator Flexibility

The high efficiency of domain-specific hardware accelerators for machine learning (ML) has come fromspecialization, with the trade-off of less configurability/ flexibility. There is growing interest in developingflexible ML accelerators to make them future-proof to the rapid evolution of Deep Neural Networks (DNNs). However, the notion of accelerator flexibility has always been used in an informal manner, restricting computer architects from conducting systematic apples-to-apples design-space exploration (DSE) across trillions of choices. In this work, we formally define accelerator flexibility and show how it can be integrated for DSE. % flows. Specifically, we capture DNN accelerator flexibility across four axes: %the map-space of DNN accelerator along four flexibility axes: tiling, ordering, parallelization, and array shape. We categorize existing accelerators into 16 classes based on their axes of flexibility support, and define a precise quantification of the degree of flexibility of an accelerator across each axis. We leverage these to develop a novel flexibility-aware DSE framework. %It respects the difference of accelerator flexibility classes and degree of flexibility support in different accelerators, creating unique map-spaces. %and forms a unique map space for exploration. % We demonstrate how this can be used to perform first-of-their-kind evaluations, including an isolation study to identify the individual impact of the flexibility axes. We demonstrate that adding flexibility features to a hypothetical DNN accelerator designed in 2014 improves runtime on future (i.e., present-day) DNNs by 11.8x geomean.

ANN-Based Inverse Goal-Oriented Design Method for Targeted Final Properties of Materials

Designing materials for targeted materials properties is the key to tackle the demands for personalized consumer products. The deficiency in the existing linear and nonlinear correlation methods attributed to simplifying assumptions and idealizations, nondeterministic simulations, and limited experimental data due to heavy computational time and cost, necessitates a design method that provides sufficient confidence to designers in decision making. To address this requirement, we propose, in this paper, an inverse goal-oriented materials design method supported by the design space exploration framework (DSEF). Keeping in view the accuracy and precision in the prediction confidence of machine learning-based methods, we developed an Artificial Neural Network based prediction model that supports DSEF. The proposed method for materials design can help designers to (1) explore PSPP spaces starting from end property requirements, (2) adjust the errors being propagated in the PSPP chain as well as in the predictions made by the model, and (3) timely adjust model parameters of the prediction model for accurate predictions. The efficacy of the method is illustrated for the hot stamping process to produce structural components from ultrahigh-strength steels (UHSS). The proposed method and prediction model are generic and applicable to any sequential manufacturing process to realize an end product.

Sherlock: A Multi-Objective Design Space Exploration Framework

Design space exploration (DSE) provides intelligent methods to tune the large number of optimization parameters present in modern FPGA high-level synthesis tools. High-level synthesis parameter tuning is a time-consuming process due to lengthy hardware compilation times—synthesizing an FPGA design can take tens of hours. DSE helps find an optimal solution faster than brute-force methods without relying on designer intuition to achieve high-quality results. Sherlock is a DSE framework that can handle multiple conflicting optimization objectives and aggressively focuses on finding Pareto-optimal solutions. Sherlock integrates a model selection process to choose the regression model that helps reach the optimal solution faster. Sherlock designs a strategy based around the multi-armed bandit problem, opting to balance exploration and exploitation based on the learned and expected results. Sherlock can decrease the importance of models that do not provide correct estimates, reaching the optimal design faster. Sherlock is capable of tailoring its choice of regression models to the problem at hand, leading to a model that best reflects the application design space. We have tested the framework on a large dataset of FPGA design problems and found that Sherlock converges toward the set of optimal designs faster than similar frameworks.

Fast Resource and Timing Aware Design Optimisation for High-Level Synthesis

Field-Programmable Gate Arrays (FPGA) are often present in energy-efficient systems, although its non-trivial development flow is an obstacle for massive adoption. High-Level Synthesis (HLS) approaches attempt to mitigate the gap by targetting FPGAs from software languages, however manual tuning is still essential to meet performance demands. We present a high-level design space exploration framework with timing and resource awareness that uses an estimator named Lina to evaluate each design point. Lina is a profiling-based approach that avoids the costly static analyses performed by HLS compilers, allowing a significantly faster exploration of optimisations. Estimations are improved by supporting a continuous range of operating frequencies and by considering resource usage for both floating-point and integer datapaths. For a given set of C kernels, the estimated solutions are among the best 1% for execution time and resource footprint. The exploration of each kernel using Lina was performed on average two orders of magnitude faster than using early HLS compiler reports, and four orders of magnitude faster than fully compiling each design point. By considering the design spaces traversed, our solutions reached 70% of the maximum speed-up achievable. This represents an average speed-up of 14-16× compared to the baseline designs with no optimisations enabled.

Enabling the potential of hybrid electric propulsion through lean-burn-combustion turbofans

Hybrid-electric propulsion has emerged as a promising technology to mitigate the adverse environmental impact of civil aviation. Boosting conventional gas turbines with electric power improves mission performance and operability. In this work the impact of electrification on pollutant emissions and direct operating cost of geared turbofan configurations is evaluated for an 150-passenger aircraft. A baseline two-and-a-half-shaft geared turbofan, representative of year 2035 entry-into-service technology, is employed. Parallel hybridization is implemented through coupling a battery-powered electric motor to the engine low-speed shaft. A multi-disciplinary design space exploration framework is employed comprising modelling methods for multi-point engine design, aircraft sizing, performance and pollutant emissions, mission and economic analysis. A probabilistic approach is developed considering uncertainties in the evaluation of direct operating cost. Sensitivities to electrical power system technology levels, as well as fuel price and emissions taxation are quantified at different time-frames. The benefits of lean direct injection are explored along short-, medium-, and long-range missions, demonstrating 32% NO&lt;italic&gt;&lt;sub&gt;x&lt;/sub&gt;&lt;/italic&gt; savings compared to traditional rich-burn, quick-mix, lean-burn technologies in short-range operations. The impact of electrification on the enhancement of lean direct injection benefits is investigated. For hybrid-electric powerplants, the take-off-to-cruise turbine entry temperature ratio is 2.5% lower than the baseline, extending the corresponding NO&lt;italic&gt;&lt;sub&gt;x&lt;/sub&gt;&lt;/italic&gt; reductions to the level of 46% in short-range missions. This work sheds light on the environmental and economic potential and limitations of a hybrid-electric propulsion concept towards a greener and sustainable civil aviation.

System-level Early-stage Modeling and Evaluation of IVR-assisted Processor Power Delivery System

Despite being employed in numerous efforts to improve power delivery efficiency, the integrated voltage regulator (IVR) approach has yet to be evaluated rigorously and quantitatively in a full power delivery system (PDS) setting. To fulfill this need, we present a system-level modeling and design space exploration framework called Ivory for IVR-assisted power delivery systems. Using a novel modeling methodology, it can accurately estimate power delivery efficiency, static performance characteristics, and dynamic transient responses under different load variations and external voltage/frequency scaling conditions. We validate the model over a wide range of IVR topologies with silicon measurement and SPICE simulation. Finally, we present two case studies using architecture-level performance and power simulators. The first case study focuses on optimal PDS design for multi-core systems, which achieves 8.6% power efficiency improvement over conventional off-chip voltage regulator module– (VRM) based PDS. The second case study explores the design tradeoffs for IVR-assisted PDSs in CPU and GPU systems with fast per-core dynamic voltage and frequency scaling (DVFS). We find 2 μs to be the optimal DVFS timescale, which not only reaps energy benefits (12.5% improvement in CPU and 50.0% improvement in GPU) but also avoids costly IVR overheads.

Deep learning framework for material design space exploration using active transfer learning and data augmentation

Neural network-based generative models have been actively investigated as an inverse design method for finding novel materials in a vast design space. However, the applicability of conventional generative models is limited because they cannot access data outside the range of training sets. Advanced generative models that were devised to overcome the limitation also suffer from the weak predictive power on the unseen domain. In this study, we propose a deep neural network-based forward design approach that enables an efficient search for superior materials far beyond the domain of the initial training set. This approach compensates for the weak predictive power of neural networks on an unseen domain through gradual updates of the neural network with active transfer learning and data augmentation methods. We demonstrate the potential of our framework with a grid composite optimization problem that has an astronomical number of possible design configurations. Results show that our proposed framework can provide excellent designs close to the global optima, even with the addition of a very small dataset corresponding to less than 0.5% of the initial training dataset size.

Design Space Exploration for Secure IoT Devices and Cyber-Physical Systems

With the advent of the Internet of Things (IoT) and Cyber-Physical Systems (CPS), embedded devices have been gaining importance in our daily lives, as well as industrial processes. Independent of their usage, be it within an IoT system or a CPS, embedded devices are always an attractive target for security attacks, mainly due to their continuous network availability and the importance of the data they handle. Thus, the design of such systems requires a thorough consideration of the various security constraints they are liable to. Introducing these security constraints, next to other requirements, such as power consumption, and performance increases the number of design choices a system designer must consider. As the various constraints are often conflicting with each other, designers face the complex task of balancing them. System designers facilitate Design Space Exploration (DSE) tools to support a system designer in this job. However, available DSE tools only offer a limited way of considering security constraints during the design process. In this article, we introduce a novel DSE framework, which allows the consideration of security constraints, in the form of attack scenarios, and attack mitigations in the form of security tasks. Based on the descriptions of the system’s functionality and architecture, possible attacks, and known mitigation techniques, the framework finds the optimal design for a secure IoT device or CPS. Our framework’s functionality and its benefits are shown based on the design of a secure sensor system.

ZigZag: Enlarging Joint Architecture-Mapping Design Space Exploration for DNN Accelerators

Building efficient embedded deep learning systems requires a tight co-design between DNN algorithms, hardware, and algorithm-to-hardware mapping, a.k.a. dataflow. However, owing to the large joint design space, finding an optimal solution through physical implementation becomes infeasible. To tackle this problem, several design space exploration (DSE) frameworks have emerged recently, yet they either suffer from long runtimes or a limited exploration space. This article introduces ZigZag, a rapid DSE framework for DNN accelerator architecture and mapping. ZigZag extends the common DSE with uneven mapping opportunities and smart mapping search strategies. Uneven mapping decouples operands (W/I/O), memory hierarchy, and mappings (temporal/spatial), opening up a whole new space for DSE, and thus better design points are found by ZigZag compared to other SotAs. For this, ZigZag uses an enhanced nested-for-loop format as a uniform representation to integrate algorithm, accelerator, and algorithm-to-accelerator mapping. ZigZag consists of three key components: 1) an analytical energy-performance-area Hardware Cost Estimator, 2) two Mapping Search Engines that support spatial and temporal even/uneven mapping search, and 3) an Architecture Generator that auto-explores the wide memory hierarchy design space. Benchmarking experiments against published works, in-house accelerator, and existing DSE frameworks, together with three case studies, show the reliability and capability of ZigZag. Up to 64 percent more energy-efficient solutions are found compared to other SotAs, due to ZigZag's uneven mapping capabilities.