Resource-constrained Hardware Platform Research Articles

TransCODE: Co-Design of Transformers and Accelerators for Efficient Training and Inference

Automated co-design of machine learning models and evaluation hardware is critical for efficiently deploying such models at scale. Despite the state-of-the-art performance of transformer models, they are not yet ready for execution on resource-constrained hardware platforms. High memory requirements and low parallelizability of the transformer architecture exacerbate this problem. Recently-proposed accelerators attempt to optimize the throughput and energy consumption of transformer models. However, such works are either limited to a one-sided search of the model architecture or a restricted set of off-the-shelf devices. Furthermore, previous works only accelerate model inference and not training, which incurs substantially higher memory and compute resources, making the problem even more challenging. To address these limitations, this work proposes a dynamic training framework, called DynaProp, that speeds up the training process and reduces memory consumption. DynaProp is a low-overhead pruning method that prunes activations and gradients at runtime. To effectively execute this method on hardware for a diverse set of transformer architectures, we propose ELECTOR, a framework that simulates transformer inference and training on a design space of accelerators. We use this simulator in conjunction with the proposed co-design technique, called TransCODE, to obtain the best-performing models with high accuracy on the given task and minimize latency, energy consumption, and chip area. The obtained transformer-accelerator pair achieves 0.3% higher accuracy than the state-of-the-art pair while incurring 5.2× lower latency and 3.0× lower energy consumption.

Implementation and Evaluation of Advantage Actor-Critic Algorithm on a Desktop Computer with a Multi-Core CPU

In this paper, the implementation and evaluation of the Advantage Actor-Critic (A2C) algorithm, one of the most important Deep Reinforcement Learning schemes, is performed. The objective is to determine the behavior of the algorithm on a desktop computer with a multi-core CPU, establishing its behavior, performance, and resource consumption for embedded applications. This algorithm makes use of multiple agents on parallel instances of the environment so that each agent adds knowledge to the system, which is weighted by a value of Advantage that evaluates its interaction in the environment. This assessment is performed on OpenAI's CartPole-v0 playground, so the results are comparable and easily reproducible. The results show a high performance of the algorithm for different instances with fixed-length segments of experience, which allows us to think of successful use on more resource-constrained hardware platforms.

Efficient Hardware Design and Implementation of the Voting Scheme-Based Convolution.

Due to a point cloud’s sparse nature, a sparse convolution block design is necessary to deal with its particularities. Mechanisms adopted in computer vision have recently explored the advantages of data processing in more energy-efficient hardware, such as the FPGA, as a response to the need to run these algorithms on resource-constrained edge devices. However, implementing it in hardware has not been properly explored, resulting in a small number of studies aimed at analyzing the potential of sparse convolutions and their efficiency on resource-constrained hardware platforms. This article presents the design of a customizable hardware block for the voting convolution. We carried out an in-depth analysis to determine under which conditions the use of the voting scheme is justified instead of dense convolutions. The proposed hardware design achieves an energy consumption about 8.7 times lower than similar works in the literature by ignoring unnecessary arithmetic operations with null weights and leveraging data dependency. Access to data memory was also reduced to the minimum necessary, leading to improvements of around 55% in processing time. To evaluate both the performance and applicability of the proposed solution, the voting convolution was integrated into the well-known PointPillars model, where it achieves improvements between 23.05% and 80.44% without a significant effect on detection performance.

A novel approach for sEMG gesture recognition using resource-constrained hardware platforms

Classifying human gestures using surface electromyografic sensors (sEMG) is a challenging task. Wearable sensors have proven to be extremely useful in this context, but their performance is limited by several factors (signal noise, computing resources, battery consumption, etc.). In particular, computing resources impose a limitation in many application scenarios, in which lightweight classification approaches are desirable. Recent research has shown that machine learning techniques are useful for human gesture classification once their salient features have been determined. This paper presents a novel approach for human gesture classification in which two different strategies are combined: a) a technique based on autoencoders is used to perform feature extraction; b) two alternative machine learning algorithms (namely J48 and K*) are then used for the classification stage. Empirical results are provided, showing that for limited computing power platforms our approach outperforms other alternative methodologies.

OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantization

As Deep Neural Networks (DNNs) usually are overparameterized and have millions of weight parameters, it is challenging to deploy these large DNN models on resource-constrained hardware platforms, e.g., smartphones. Numerous network compression methods such as pruning and quantization are proposed to reduce the model size significantly, of which the key is to find suitable compression allocation (e.g., pruning sparsity and quantization codebook) of each layer. Existing solutions obtain the compression allocation in an iterative/manual fashion while finetuning the compressed model, thus suffering from the efficiency issue. Different from the prior art, we propose a novel One-shot Pruning-Quantization (OPQ) in this paper, which analytically solves the compression allocation with pre-trained weight parameters only. During finetuning, the compression module is fixed and only weight parameters are updated. To our knowledge, OPQ is the first work that reveals pre-trained model is sufficient for solving pruning and quantization simultaneously, without any complex iterative/manual optimization at the finetuning stage. Furthermore, we propose a unified channel-wise quantization method that enforces all channels of each layer to share a common codebook, which leads to low bit-rate allocation without introducing extra overhead brought by traditional channel-wise quantization. Comprehensive experiments on ImageNet with AlexNet/MobileNet-V1/ResNet-50 show that our method improves accuracy and training efficiency while obtains significantly higher compression rates compared to the state-of-the-art.

Bayesian tensorized neural networks with automatic rank selection

Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In order to address this challenge, this paper proposes a low-rank Bayesian tensorized neural network. Our Bayesian method performs automatic model compression via an adaptive tensor rank determination. We also present approaches for posterior density calculation and maximum a posteriori (MAP) estimation for the end-to-end training of our tensorized neural network. We provide experimental validation on a two-layer fully connected neural network, a 6-layer CNN and a 110-layer residual neural network where our work produces 7.4× to 137× more compact neural networks directly from the training while achieving high prediction accuracy.

Open-Source Heterogeneous Constrained Edge-Computing Platform for Smart Grid Measurements

This article presents a low-cost, open-source, heterogeneous, resource-constrained hardware platform called “Parallella” as a measurement device for edge-computing applications research in smart grid. The unique hardware architecture of the Parallella provides a multitude of edge-computing resources in the form of a Zynq SoC (dual-core ARM + FPGA) and a 16-core co-processor called Epiphany. A multifunctional intelligent electronic device (IED) design is demonstrated to showcase the capabilities of the platform. A custom I/O board has been developed for the desktop and embedded versions of Parallella, which can be interfaced with external daughter boards and peripherals for measurements. One such daughter board is an analog sensing board, which can measure voltages of all the three phases and four line currents using a 16-bit synchronous ADC set at 32 kHz. The ADC samples are synchronized to the PPS time clock of a GPS unit for providing global time reference. These captured seven-channel raw waveform data are sent to a cloud server over a bandwidth-limited communication channel using a custom anomaly-aware data compression algorithm implemented on the ARM. A phasor measurement algorithm using the Teager energy operator (TEO) is implemented on the field-programmable gate array (FPGA). A parallel power quality (PQ) measurement algorithm is implemented on the Epiphany. The obtained measurements are found to be comparable to a commercial power analyzer.

Hardware design and the competency awareness of a neural network

The ability to estimate the uncertainty of predictions made by a neural network is essential when applying neural networks to tasks such as medical diagnosis and autonomous vehicles. The approach is of particular relevance when deploying the networks on devices with limited hardware resources, but existing competency-aware neural networks largely ignore any resource constraints. Here we examine the relationship between hardware platforms and the competency awareness of a neural network. We highlight the impact of two key areas of hardware development — increasing memory size of accelerator architectures and device-to-device variation in the emerging devices typically used in in-memory computing — on uncertainty estimation quality. We also consider the challenges that developments in uncertainty estimation methods impose on hardware designs. Finally, we explore the innovations required in terms of hardware, software, and hardware–software co-design in order to build future competency-aware neural networks. This Perspective examines the relationship between hardware platforms and the competency awareness of a neural network, highlighting how hardware developments can impact uncertainty estimation quality, and exploring the innovations required in order to build competency-aware neural networks in resource constrained hardware platforms.

Survey on Deep Neural Networks in Speech and Vision Systems

This survey presents a review of state-of-the-art deep neural network architectures, algorithms, and systems in speech and vision applications. Recent advances in deep artificial neural network algorithms and architectures have spurred rapid innovation and development of intelligent speech and vision systems. With availability of vast amounts of sensor data and cloud computing for processing and training of deep neural networks, and with increased sophistication in mobile and embedded technology, the next-generation intelligent systems are poised to revolutionize personal and commercial computing. This survey begins by providing background and evolution of some of the most successful deep learning models for intelligent speech and vision systems to date. An overview of large-scale industrial research and development efforts is provided to emphasize future trends and prospects of intelligent speech and vision systems. Robust and efficient intelligent systems demand low-latency and high fidelity in resource-constrained hardware platforms such as mobile devices, robots, and automobiles. Therefore, this survey also provides a summary of key challenges and recent successes in running deep neural networks on hardware-restricted platforms, i.e. within limited memory, battery life, and processing capabilities. Finally, emerging applications of speech and vision across disciplines such as affective computing, intelligent transportation, and precision medicine are discussed. To our knowledge, this paper provides one of the most comprehensive surveys on the latest developments in intelligent speech and vision applications from the perspectives of both software and hardware systems. Many of these emerging technologies using deep neural networks show tremendous promise to revolutionize research and development for future speech and vision systems.

Performance Assessment of Deep Learning Frameworks through Metrics of CPU Hardware Exploitation on an Embedded Platform

In this paper, we analyze heterogeneous performance exhibited by some popular deep learning software frameworks for visual inference on a resource-constrained hardware platform. Benchmarking of Caffe, OpenCV, TensorFlow, and Caffe2 is performed on the same set of convolutional neural networks in terms of instantaneous throughput, power consumption, memory footprint, and CPU utilization. To understand the resulting dissimilar behavior, we thoroughly examine how the resources in the processor are differently exploited by these frameworks. We demonstrate that a strong correlation exists between hardware events occurring in the processor and inference performance. The proposed hardware-aware analysis aims to find limitations and bottlenecks emerging from the joint interaction of frameworks and networks on a particular CPU-based platform. This provides insight into introducing suitable modifications in both types of components to enhance their global performance. It also facilitates the selection of frameworks and networks among a large diversity of these components available these days for visual understanding.

Lucid: A Practical, Lightweight Deep Learning Solution for DDoS Attack Detection

Distributed Denial of Service (DDoS) attacks are one of the most harmful threats in today's Internet, disrupting the availability of essential services. The challenge of DDoS detection is the combination of attack approaches coupled with the volume of live traffic to be analysed. In this paper, we present a practical, lightweight deep learning DDoS detection system called LUCID, which exploits the properties of Convolutional Neural Networks (CNNs) to classify traffic flows as either malicious or benign. We make four main contributions; (1) an innovative application of a CNN to detect DDoS traffic with low processing overhead, (2) a dataset-agnostic preprocessing mechanism to produce traffic observations for online attack detection, (3) an activation analysis to explain LUCID's DDoS classification, and (4) an empirical validation of the solution on a resource-constrained hardware platform. Using the latest datasets, LUCID matches existing state-of-the-art detection accuracy whilst presenting a 40x reduction in processing time, as compared to the state-of-the-art. With our evaluation results, we prove that the proposed approach is suitable for effective DDoS detection in resource-constrained operational environments.

Internet of things-based real-time model study on e-healthcare: Device, message service and dew computing

Real-timeliness is the key aspect for the growing perspective of Internet of Things (IoT). IoT being a heterogenous and highly distributed framework requires instantaneous adoption of collateral attributes. Due to the lack of service-oriented manifestation of timely signaling, may hinder back the IoT's envisaged speculation. This paper investigates three possible solutions where IoT can be benefited with real-time affect that includes (i) multi-tasking of popular micro-processing modules, (ii) sensor-generated IoT stream processing in cloud-centric medium and (iii) dew computing-based context-aware local computing. The objective of this article is to provide an in-depth analysis and validation of real-time scheduling in resource constrained hardware platforms through real-time message passing in presence of a cloud facilitator with a real-time context. A novel dew computing architecture is hereby introduced and proposed to foresee as an enabler of the context-aware service framework in purely real-time fashion. The promising outcomes validated the proposed frameworks when compared to existing alternatives in terms of efficiency, responsiveness and appropriateness. All the three studies are concentrated to envisage the possible impact on the IoT e-healthcare as a model study. The discussion section presents an overall analysis of the three deployed experiments to find their pros and cons.

Adequation and hardware implementation of the color structure descriptor for real-time temporal video segmentation

In this paper, we focused on the study of the MPEG-7 color structure descriptor (CSD) for real-time temporal video segmentation. A new hardware architecture of the CSD was proposed. In the aim of optimizing this implementation in terms of hardware resources and execution time, different algorithm transformations have been tested for the considered application. The CSD was applied for different quantization levels in the HMMD color space and for different grey levels, with and without a frame skip. We have demonstrated that the use of a low number of quantization levels and a frame skip can significantly reduce the complexity and assure a better computing performance while preserving a satisfactory level of accuracy in terms of shot boundary detection rate. This is useful for implementation on resource-constrained hardware platform and multiprocessing applications. The performances of the proposed architecture were evaluated for different quantization levels to show the effect on occupied hardware resources and execution time. The comparative study demonstrates the effectiveness of the proposed architecture, which can operate in real-time video and without restriction on image size. In this work we have also designed a system on chip (SOC) for real-time video summarization based on the CSD. The proposed SOC integrating the CSD module was developed using a platform based on a Xilinx Virtex5 FPGA. A complete demonstration, including CSD extraction, shot boundary detection and key-frames visualization, was realized.

Editorial for the special issue on search‐based software engineering

Editorial for the special issue on search‐based software engineering

Generalized neuron: Feedforward and recurrent architectures

Feedforward neural networks such as multilayer perceptrons (MLP) and recurrent neural networks are widely used for pattern classification, nonlinear function approximation, density estimation and time series prediction. A large number of neurons are usually required to perform these tasks accurately, which makes the MLPs less attractive for computational implementations on resource constrained hardware platforms. This paper highlights the benefits of feedforward and recurrent forms of a compact neural architecture called generalized neuron (GN). This paper demonstrates that GN and recurrent GN (RGN) can perform good classification, nonlinear function approximation, density estimation and chaotic time series prediction. Due to two aggregation functions and two activation functions, GN exhibits resilience to the nonlinearities of complex problems. Particle swarm optimization (PSO) is proposed as the training algorithm for GN and RGN. Due to a small number of trainable parameters, GN and RGN require less memory and computational resources. Thus, these structures are attractive choices for fast implementations on resource constrained hardware platforms.

Call for Papers: Practical Aspects of Search‐Based Software Engineering

Software systems are becoming ever larger and more complex as new architectures emerge and high-performance hardware becomes increasingly affordable. However, many of today's approaches to software design and implementation are unable to scale to meet the challenges presented by such systems. For this reason, a recent trend has been to automate tasks within the software engineering lifecycle using machine-based search. The engineering task is reformulated as an optimization problem and solutions are found using efficient modern optimization algorithms, such as meta-heuristic search and operational research (OR) methods. This approach, known as Search-Based Software Engineering (SBSE), promises much greater scalability than traditional labour-intensive methods since human effort is redirected to guide the search for solutions to the engineering problem, rather than to perform the search itself. SBSE has been applied across the spectrum of software engineering activities, including requirements engineering, project planning, software task allocation, code refactoring, protocol synthesis, test data generation, and the design of algorithms for highly resource-constrained hardware platforms. As the efficacy and scalability of this approach is established for an increasing number of software engineering problems, a particular research focus is how to successfully apply SBSE in practice. Special Issue Scope This special issue of Software—Practice and Experience invites the submission of original manuscripts describing applied research into the practical aspects of SBSE. We particularly welcome case studies and other work demonstrating practical experience of using SBSE to solve real-world or benchmark problems. The engineering problems solved may relate to any aspect of software design and implementation, and the search techniques used may include (but are not limited to) meta-heuristic search algorithms, OR methods, and hybrid algorithms that combine more than one technique. Submission The deadline for submissions is August 14, 2009. Manuscripts must conform to the submission guidelines for Software—Practice and Experience. For information please see the ‘For Authors’ link on the journal homepage at: http://www3.interscience.wiley.com/journal/1752/home. Manuscripts must not have been published previously or be currently under consideration for publication in any other journal or conference. Significant extensions to substantive papers published in conference proceedings are welcome and must be accompanied by a letter that details the additional material in the submission to this special issue. Please submit your manuscript using the online submission system, as described in the ‘instructions for authors’ referenced above. During the submission process you should indicate that the manuscript is for a special issue, and you will be asked to supply the name of the special issue, which is: ‘Practical Aspects of Search-Based Software Engineering’.

Real-Time Image and Video Processing: From Research to Reality

This book presents an overview of the guidelines and strategies for transitioning an image or video processing algorithm from a research environment into a real-time constrained environment. Such guidelines and strategies are scattered in the literature of various disciplines including image processing, computer engineering, and software engineering, and thus have not previously appeared in one place. By bringing these strategies into one place, the book is intended to serve the greater community of researchers, practicing engineers, industrial professionals, who are interested in taking an image or video processing algorithm from a research environment to an actual real-time implementation on a resource constrained hardware platform. These strategies consist of algorithm simplifications, hardware architectures, and software methods. Throughout the book, carefully selected representative examples from the literature are presented to illustrate the discussed concepts. After reading the book, the readers are exposed to a wide variety of techniques and tools, which they can then employ to design a real-time image or video processing system.