ARM Cortex Research Articles

Efficient and portable Winograd convolutions for multi-core processors

We take a step forward towards developing high-performance codes for the convolution operator, based on the Winograd algorithm, that are easy to customise for general-purpose processor architectures. In our approach, augmenting the portability of the solution is achieved via the introduction of vector instructions from Intel SSE/AVX2/AVX512 and ARM NEON/SVE to exploit the single-instruction multiple-data capabilities of current processors as well as OpenMP pragmas to exploit multi-threaded parallelism. While this comes at the cost of sacrificing a fraction of the computational performance, our experimental results on three distinct processors, with Intel Xeon Skylake, ARM Cortex A57 and Fujitsu A64FX processors, show that the impact is affordable and still renders a Winograd-based solution that is competitive when compared with the lowering gemm-based convolution.

A fully-mapped and energy-efficient FPGA accelerator for dual-function AI-based analysis of ECG.

In this paper, a fully-mapped field programmable gate array (FPGA) accelerator is proposed for artificial intelligence (AI)-based analysis of electrocardiogram (ECG). It consists of a fully-mapped 1-D convolutional neural network (CNN) and a fully-mapped heart rate estimator, which constitute a complementary dual-function analysis. The fully-mapped design projects each layer of the 1-D CNN to a hardware module on an Intel Cyclone V FPGA, and a virtual flatten layer is proposed to effectively bridge the feature extraction layers and fully-connected layer. Also, the fully-mapped design maximizes computational parallelism to accelerate CNN inference. For the fully-mapped heart rate estimator, it performs pipelined transformations, self-adaptive threshold calculation, and heartbeat count on the FPGA, without multiplexed usage of hardware resources. Furthermore, heart rate calculation is elaborately analyzed and optimized to remove division and acceleration, resulting in an efficient method suitable for hardware implementation. According to our experiments on 1-D CNN, the accelerator can achieve 43.08× and 8.38× speedup compared with the software implementations on ARM-Cortex A53 quad-core processor and Intel Core i7-8700 CPU, respectively. For the heart rate estimator, the hardware implementations are 25.48× and 1.55× faster than the software implementations on the two aforementioned platforms. Surprisingly, the accelerator achieves an energy efficiency of 63.48 GOPS/W, which obviously surpasses existing studies. Considering its power consumption is only 67.74mW, it may be more suitable for resource-limited applications, such as wearable and portable devices for ECG monitoring.

A 16-nm SoC for Noise-Robust Speech and NLP Edge AI Inference With Bayesian Sound Source Separation and Attention-Based DNNs

The proliferation of personal artificial intelligence (AI) -assistant technologies with speech-based conversational AI interfaces is driving the exponential growth in the consumer Internet of Things (IoT) market. As these technologies are being applied to keyword spotting (KWS), automatic speech recognition (ASR), natural language processing (NLP), and text-to-speech (TTS) applications, it is of paramount importance that they provide uncompromising performance for context learning in long sequences, which is a key benefit of the attention mechanism, and that they work seamlessly in polyphonic environments. In this work, we present a 25-mm2 system-on-chip (SoC) in 16-nm FinFET technology, codenamed SM6, which executes end-to-end speech-enhancing attention-based ASR and NLP workloads. The SoC includes: 1) FlexASR, a highly reconfigurable NLP inference processor optimized for whole-model acceleration of bidirectional attention-based sequence-to-sequence (seq2seq) deep neural networks (DNNs); 2) a Markov random field source separation engine (MSSE), a probabilistic graphical model accelerator for unsupervised inference via Gibbs sampling, used for sound source separation; 3) a dual-core Arm Cortex A53 CPU cluster, which provides on-demand single Instruction/multiple data (SIMD) fast fourier transform (FFT) processing and performs various application logic (e.g., expectation–maximization (EM) algorithm and 8-bit floating-point (FP8) quantization); and 4) an always-ON M0 subsystem for audio detection and power management. Measurement results demonstrate the efficiency ranges of 2.6–7.8 TFLOPs/W and 4.33–17.6 Gsamples/s/W for FlexASR and MSSE, respectively; MSSE denoising performance allowing 6 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> smaller ASR model to be stored on-chip with negligible accuracy loss; and 2.24-mJ energy consumption while achieving real-time throughput, end-to-end, and per-frame ASR latencies of 18 ms.

A Lightweight Architecture for Query-by-Example Keyword Spotting on Low-Power IoT Devices

Keyword spotting (KWS) is a task to recognize a keyword or a particular command in a continuous audio stream, which can be effectively applied to a voice trigger system that automatically monitors and processes speech signals. This paper focuses on the problem of user-defined keyword spotting in low-resource settings. A lightweight neural network architecture is developed for tackling the keyword detection task using query-by-example (QbyE) techniques. The architecture uses a convolutional recurrent neural network (CRNN) to extract the frame-level features of input audio signals. A customized model compression method is proposed to compress the network, making it suitable for low power settings. In the keyword enrollment, all enrolled keyword examples are merged to generate a single keyword template, which is responsible for detecting a target keyword in keyword search. To improve the efficiency of keyword searching, a segmental local normalized DTW algorithm is introduced. Experiments on the real-world collected datasets show that our approach consistently outperforms the state-of-the-art methods, and the proposed system can run on an ARM Cortex-A7 processor and achieve real-time keyword detection.

Performance–energy trade-offs of deep learning convolution algorithms on ARM processors

In this work, we assess the performance and energy efficiency of high-performance codes for the convolution operator, based on the direct, explicit/implicit lowering and Winograd algorithms used for deep learning (DL) inference on a series of ARM-based processor architectures. Specifically, we evaluate the NVIDIA Denver2 and Carmel processors, as well as the ARM Cortex-A57 and Cortex-A78AE CPUs as part of a recent set of NVIDIA Jetson platforms. The performance–energy evaluation is carried out using the ResNet-50 v1.5 convolutional neural network (CNN) on varying configurations of convolution algorithms, number of threads/cores, and operating frequencies on the tested processor cores. The results demonstrate that the best throughput is obtained on all platforms with the Winograd convolution operator running on all the cores at their highest frequency. However, if the goal is to reduce the energy footprint, there is no rule of thumb for the optimal configuration.

OpTIFlow &amp;#x2013; An optimized end-to-end dataflow for accelerating deep learning workloads on heterogeneous SoCs

A typical edge compute SoC capable of handling deep learning workloads at low power is usually heterogeneous by design. It typically comprises multiple initiators such as real-time IPs for capture and display, hardware accelerators for ISP, computer vision, deep learning engines, codecs, DSP or ARM cores for general compute, GPU for 2D/3D visualization. Every participating initiator transacts with common resources such as L3/L4/DDR memory systems to seamlessly exchange data between them. A careful orchestration of this dataflow is important to keep every producer/consumer at full utilization without causing any drop in real-time performance which is critical for automotive applications. The software stack for such complex workflows can be quite intimidating for customers to bring-up and more often act as an entry barrier for many to even evaluate the device for performance. In this paper we propose techniques developed on TI&#x2019;s latest TDA4V-Mid SoC, targeted for ADAS and autonomous applications, which is designed around ease-of-use but ensuring device entitlement class of performance using open standards such as DL runtimes, OpenVx and GStreamer.

FPGA Implementation of the Chirp-Scaling Algorithm for Real-Time Synthetic Aperture Radar Imaging.

Synthetic aperture radar (SAR), which can generate images of regions or objects, is an important research area of radar. The chirp scaling algorithm (CSA) is a representative SAR imaging algorithm. The CSA has a simple structure comprising phase compensation and fast Fourier transform (FFT) operations by replacing interpolation for range cell migration correction (RCMC) with phase compensation. However, real-time processing still requires many computations and a long execution time. Therefore, it is necessary to develop a hardware accelerator to improve the speed of algorithm processing. In addition, the demand for a small SAR system that can be mounted on a small aircraft or drone and that satisfies the constraints of area and power consumption is increasing. In this study, we proposed a CSA-based SAR processor that supports FFT and phase compensation operations and presents field-programmable gate array (FPGA)-based implementation results. We also proposed a modified CSA flow that simplifies the traditional CSA flow by changing the order in which the transpose operation occurs. Therefore, the proposed CSA-based SAR processor was designed to be suitable for modified CSA flow. We designed the multiplier for FFT to be shared for phase compensation, thereby achieving area efficiency and simplifying the data flow. The proposed CSA-based SAR processor was implemented on a Xilinx UltraScale+ MPSoC FPGA device and designed using Verilog-HDL. After comparing the execution times of the proposed SAR processor and the ARM cortex-A53 microprocessor, we observed a 136.2-fold increase in speed for the 4096 × 4096-pixel image.

Implementation of Floating‐Point Arithmetic Processing on Content Addressable Memory‐Based Massive‐Parallel SIMD matriX Core

AbstractSeveral multimedia applications have recently been implemented on mobile devices, including digital image compression, video compression, and audio processing. Furthermore, Artificial Intelligence (AI) processing has grown in popularity, necessitating the execution of large amounts of data in mobile devices. Therefore, the processing core in a mobile device requires high performance, programmability, and versatility. Multimedia apps for mobile devices typically comprise repeated arithmetic and table‐lookup coding operations. A Content Addressable Memory‐based massive‐parallel SIMD matriX core (CAMX) is presented to increase the processing speed of both operations on a processing core. The CAMX serves as a CPU core accelerator for mobile devices. The CAMX supports high‐parallel processing and is equipped with two CAM modules for high‐speed repeated arithmetic and table‐lookup coding operations. The CAMX has great performance, programmability, and versatility on mobile devices because it can handle logical, arithmetic, search, and shift operations in parallel. This paper shows that the CAMX can process parallel repeated arithmetic and table‐lookup coding operations; single‐precision floating‐point arithmetic can calculate 1024 entries in 5613 clock cycles in parallel without embedding a dedicated floating‐point arithmetic unit. This clock cycle using two's complement‐reduced floating‐point addition implementation decreases 59% than the implementation of straight‐forward floating‐point addition. The implementation of straight‐forward floating‐point additions is improved as two's complement instruction reduced algorithms. Thus, this paper proposes an instruction reduction architecture by modulating the CAMX to directly access the data in the left and right CAM modules from the preserve register. The CAMX has achieved high performance, programmability, and versatility by not embedding a dedicated processing unit. Moreover, assuming the CAMX processes at an operating frequency of 0.1, 0.5, 1.0, or 1.5 GHz, it can process floating‐point additions above approximately 4500 parallelized data, with better performance than an ARM core using NEON and Vector Floating‐Point (VFP). In addition, related works executed by software instruction, dedicated floating‐point arithmetic unit, or both and the CAMX are compared while assuming the same operation frequency. From this result, the CAMX which has 128‐bit and 1024‐entry CAM modules achieves higher performance than the related works executed by only software instructions and by combining software instructions and a dedicated floating‐point arithmetic unit. © 2023 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Role of 5G Networks in Healthcare Management System.

The present-day healthcare system operates on a 4G network, where the data rate needed for many IoT devices is impossible. Also, the latency involved in the network does not support the use of many devices in the network. The 5G-based cellular technology promises an effective healthcare management system with high speed and low latency. The 5G communication technology will replace the 4G technology to satisfy the increasing demand for high data rates. It incorporates higher frequency bands of around 100 MHz using millimetre waves and broadband modulation schemes. It is aimed at providing low latency while supporting real-time machine-to-machine communication. It requires a more significant number of antennas, with an average base station density three times higher than 4G. However, the rise in circuit and processing power for multiple antennas and transceivers deteriorates energy efficiency. Also, the data transmission power for 5G is three times higher than for 4G technology. One of the advanced processors used in today's mobile equipment is NVIDIA Tegra, which has a multicore system on chip (SoC) architecture with two ARM Cortex CPU cores to handle audio, images, and video. The state-of-the-art software coding using JAVA or Python has achieved smooth data transmission from mobile equipment, desktop or laptop through the internet with the support of 5G communication technology. This paper discusses some key areas related to 5G-based healthcare systems such as the architecture, antenna designs, power consumption, file protocols, security, and health implications of 5G networks.

Design of RISC Processor with IEEE754 Standard Floating-Point Instruction Set in FPGA using VHDL for Digital Signal Processing Applications

The design of RISC processors, which are the key of digital signal processing applications, are increasing in reconfigurable hardware. FPGAs are suitable reconfigurable hardware for RISC processor design, with advantages such as parallel processing and low power consumption. In this study, the design of the 32-bit RISC processor in a FPGA is presented. The designed RISC processor contains IEEE754 standard floating-point number processing unit, which is executed in a one clock cycle. The verification of the processor is performed for the Zynq-7000 SoC Artix-7 FPGA chip in the Xilinx Vivado tool. Classification of an artificial neural network using the iris dataset is carried out in this designed RISC processor. In order to compare the performance, the same artificial neural network is executed in real time within the dual-core ARM Cortex-A9 processor in the operating system of the Zynq-7000 SoC. The results show that the RISC processor designed in the FPGA executes at 20x less clock cycles and 3x higher speed compared to the ARM processor.

RESEARCH OF A MICROCONTROLLER CHARGER FOR LITHIUM BATTERIES

Lithium battery chargers work on the principle of limiting the charge voltage. Among the features of such a charge, the following parameters are distinguished, such as a 3—4 times increase in the voltage of the lithium battery cell, smaller allowable deviations of the charge voltage (linearity of the current-voltage characteristic) and the absence of the need for a compensatory charge (jet recharging) after the battery reaches a state of full charge . Let's consider the specifics of the charge on the example of portable lithium batteries with an output voltage of 4.2 V, the specifics of the charge of which (and with a higher output voltage) consists in the use of the CC-CV method.&#x0D; To simulate the operation of the charger and debug the MK program, an electrical schematic diagram was compiled in the ISIS Proteus environment. To test this circuit, the model includes an STM32F103 MC, which is used to generate a PWM signal whose frequency is 62 kHz. Next, the output current is measured, with a setting of the PWM signal strength within 50 %. According to the indicators at the output of the operational amplifier in the current measurement mode, it is possible to determine the current in the ammeter battery circuit.&#x0D; Based on the simulation results of the developed scheme and algorithm, it can be concluded that this scheme ensures the fulfillment of the necessary modes of operation of the investigated charging device and is suitable for further implementation. A 32-bit MC of the STM32F103C8 series, located on the BluePill platform — a development board for the ARM Cortex M3 microcontroller, was selected as the control element of the control unit. The stable operating mode of the MK is ensured by two crystal oscillators of 8 MHz and 32768 Hz, and using the built-in frequency multiplier, the MK provides support for the operating frequency equal to 72 MHz.&#x0D; As a result, the functionality of the charger circuit and the SS-СV algorithm for fast charging of lithium batteries was developed and confirmed. The use of the CC-CV charge method made it possible to increase the output current of the charge, as well as reduce the time required to restore the energy properties of the battery, while current fluctuations are reduced by 96 %, compared to schemes that use only constant voltage methods.

A comparative study of hash algorithms with the prospect of developing a CAN bus authentication technique

In this paper, the performances of SHA-3 final round candidates along with new versions of other hash algorithms are analyzed and compared. An ARM-Cortex A9 microcontroller and a Spartan -3 FPGA circuit are involved in the study, with emphasis placed on the number of cycles and the authentication speed. These hash functions are implemented and tested resulting in a set of ranked algorithms in terms of the specified metrics. Taking into account the performances of the most efficient algorithms and the proposed hardware platform components, an authentication technique can be developed as a possible solution to the limitations and weaknesses of automotive CAN (Controlled Area Network) bus – based embedded systems in terms of security, privacy and integrity. From there, the main elements of such a potential structure are set forth.

Design of low cost IoT enabled greenhouse control system for precision agricultural research application

Greenhouse is climate controlled transparent chamber made of materials like polycarbonate, in which crops are grown under controlled climatic conditions. This research paper aims to present a design of low-cost IoT interfaced Greenhouse Control Unit, where microclimate is controlled using various sensors and actuators to maintain desired environmental conditions as per crop need. It is used in precision agricultural research to study the impact of environmental parameters like humidity, temperature, ambient light and soil moisture on crop yield and its nutritional components. Temperature, humidity, ambient light and soil moisture like parameters are fetched by deploying suitable sensors at suitable locations in the chamber, and based on predefined threshold values of these parameters switching of cooling, heating system, light, fogger, dehumidifiers, and irrigation system are controlled using embedded controller device and desired climatic condition is maintained inside the Greenhouse chamber for the required duration. Such control units are required for agricultural research to study the impact of environmental parameters on crop yield and on the nutritional contents of the crop. It is also used in producing medicinal and herbal plants in the region which climate is very harsh for these plants. Authors have developed a low-cost cloud interfaced standalone embedded control unit around ARM Cortex M3 CPU, various sensors for controlling greenhouse actuators based on pre-defined threshold values of environmental parameters.

Online Learning for Orchestration of Inference in Multi-user End-edge-cloud Networks

Deep-learning-based intelligent services have become prevalent in cyber-physical applications, including smart cities and health-care. Deploying deep-learning-based intelligence near the end-user enhances privacy protection, responsiveness, and reliability. Resource-constrained end-devices must be carefully managed to meet the latency and energy requirements of computationally intensive deep learning services. Collaborative end-edge-cloud computing for deep learning provides a range of performance and efficiency that can address application requirements through computation offloading. The decision to offload computation is a communication-computation co-optimization problem that varies with both system parameters (e.g., network condition) and workload characteristics (e.g., inputs). However, deep learning model optimization provides another source of tradeoff between latency and model accuracy. An end-to-end decision-making solution that considers such computation-communication problem is required to synergistically find the optimal offloading policy and model for deep learning services. To this end, we propose a reinforcement-learning-based computation offloading solution that learns optimal offloading policy considering deep learning model selection techniques to minimize response time while providing sufficient accuracy. We demonstrate the effectiveness of our solution for edge devices in an end-edge-cloud system and evaluate with a real-setup implementation using multiple AWS and ARM core configurations. Our solution provides 35% speedup in the average response time compared to the state-of-the-art with less than 0.9% accuracy reduction, demonstrating the promise of our online learning framework for orchestrating DL inference in end-edge-cloud systems.

High-order Polynomial Comparison and Masking Lattice-based Encryption

The main protection against side-channel attacks consists in computing every function with multiple shares via the masking countermeasure. For IND-CCA secure lattice-based encryption schemes, the masking of the decryption algorithm requires the high-order computation of a polynomial comparison. In this paper, we describe and evaluate a number of different techniques for such high-order comparison, always with a security proof in the ISW probing model. As an application, we describe the full high-order masking of the NIST standard Kyber, with a concrete implementation on ARM Cortex M architecture, and a t-test evaluation.

E-prop on SpiNNaker 2: Exploring online learning in spiking RNNs on neuromorphic hardware.

In recent years, the application of deep learning models at the edge has gained attention. Typically, artificial neural networks (ANNs) are trained on graphics processing units (GPUs) and optimized for efficient execution on edge devices. Training ANNs directly at the edge is the next step with many applications such as the adaptation of models to specific situations like changes in environmental settings or optimization for individuals, e.g., optimization for speakers for speech processing. Also, local training can preserve privacy. Over the last few years, many algorithms have been developed to reduce memory footprint and computation. A specific challenge to train recurrent neural networks (RNNs) for processing sequential data is the need for the Back Propagation Through Time (BPTT) algorithm to store the network state of all time steps. This limitation is resolved by the biologically-inspired E-prop approach for training Spiking Recurrent Neural Networks (SRNNs). We implement the E-prop algorithm on a prototype of the SpiNNaker 2 neuromorphic system. A parallelization strategy is developed to split and train networks on the ARM cores of SpiNNaker 2 to make efficient use of both memory and compute resources. We trained an SRNN from scratch on SpiNNaker 2 in real-time on the Google Speech Command dataset for keyword spotting. We achieved an accuracy of 91.12% while requiring only 680 KB of memory for training the network with 25 K weights. Compared to other spiking neural networks with equal or better accuracy, our work is significantly more memory-efficient. In addition, we performed a memory and time profiling of the E-prop algorithm. This is used on the one hand to discuss whether E-prop or BPTT is better suited for training a model at the edge and on the other hand to explore architecture modifications to SpiNNaker 2 to speed up online learning. Finally, energy estimations predict that the SRNN can be trained on SpiNNaker2 with 12 times less energy than using a NVIDIA V100 GPU.

Development of ARM and CPLD Based Multi Controller Board

Over the course of the last ten years, the number of researchers, academicians, and industry people participating in development boards has significantly increased. In order for product development departments to realize their objective of generating higher-quality items, it is vital for them to integrate the processes of developing both hardware and software. The fundamental concept that supports a multi-controller board which incorporates ARM and CPLD cores has been materialised into a working product. There is not yet a version of a Multi-Controller Boards that is currently available in the marketplace that incorporates both CPLD and ARM into such a single board at the same time. This is the case despite the fact that such a board would be desirable. Creating a board that can accommodate many controllers is the primary objective of this endeavor. In addition to a switching circuit, this board will feature an ARM chipset as well as a CPLD chipset as its primary components. In order to transition between the two chipsets, the switching circuitry have been utilized.

3D CNN Hardware Circuit for Motion Recognition Based on FPGA

In recent years, three-dimensional convolutional neural network (3D CNN) has been widely used in the fields of action recognition and video analysis. The general purpose processors are difficult to achieve efficient and intensive computing, and the deployment of 3D CNN based on FPGA has the advantages of low power consumption, high energy efficiency, and customizability, and has gradually become a hot choice for deploying convolutional neural networks in many embedded scenarios. This paper designs a small 3D convolutional neural network based on the classic 3D convolutional neural network C3D, and uses the general matrix multiplication (GEMM) to map the 3D convolution calculation to the 2D matrix multiplication calculation. The matrix is divided into blocks and transmitted to the FPGA through the AXI bus, and the multiplication operation of the block matrix is realized through a two-dimensional multiply-accumulate array. A System on Chip (SoC) architecture is built on the PYNQ platform, using ARM Cortex-A9 as the process control core, and the calculation of the entire matrix is completed under reasonable block and scheduling on the ARM. The IP core of matrix calculation is designed using High Level Synthesis (HLS), and the corresponding parallel optimization scheme is given. Experiments have verified that the prototype design of the hardware acceleration circuit achieves low power consumption, high energy efficiency, and high precision motion recognition while using less hardware resources.

Transformation from arm joint coordinates to hand external coordinates explains non-uniform precision of hand position sense in horizontal workspace

People perceive hand position in horizontal workspace more precisely in radial than in azimuth directions and closer to the body than farther away. Current explanations for this position sense non-uniformity include spatial asymmetry in arm proprioceptive activities and/or cortex maps, experience-dependent learning, arm posture, and others. Here we investigated contributions to this non-uniformity of a posture-dependent transformation from arm joint angles, sensed by arm proprioceptors, to hand position. We measured precision of hand position sense in a bimanual hand mirror-position matching task at four horizontal targets forming a square in front of the body in 11 blindfolded individuals. We found lower hand precision in azimuth than in radial direction, higher azimuth precision at close targets, and higher radial precision at distant targets. We then theoretically analyzed the transformation of random angle errors at shoulder and elbow into hand position random errors in a horizontal plane and obtained similar distributions of hand position errors. The predicted and experimental hand-precision ellipse orientations, but not ellipse shapes or sizes, were highly correlated and were nearly orthogonal to arm stiffness ellipse orientations reported in the literature. We concluded that the joint-to-hand coordinate transformation is responsible for the non-uniform precision of hand position sense.

Optimize Image Processing Algorithm on ARM Cortex-A72 and A53

This work presents a technique to optimize processing image algorithms. The increasing demand for video applications like context-aware computing on mobile embedded systems requires the use of computationally intensive image processing algorithms. The system engineer has the mandate to optimize them so as to meet real-time deadlines. A methodology to take advantage of the asymmetric dual-core processor, which includes an ARM supported by shared memory, is presented with implementation details. The target platform chosen is the NanoPi M4V2. It has a dual-core and quad-core architecture with an ARM Cortex-A72 and Cortex-A53. The basic image correlation algorithm is chosen for benchmarking as it finds widespread application for various template-matching tasks such as face recognition. The basic algorithm prototypes conform to OpenCV, a popular computer vision library. OpenCV algorithms can be easily ported to the ARM core, which runs a popular operating system such as Linux. The algorithms are tested on a variety of images, and performance results are presented, measuring the speedup obtained due to dual-core and quad-core implementation. A major advantage of this approach is that it allows the ARM processor to perform important real-time tasks.