Hardware Platform Research Articles

MixTrain: accelerating DNN training via input mixing.

Training Deep Neural Networks (DNNs) places immense compute requirements on the underlying hardware platforms, expending large amounts of time and energy. An important factor contributing to the long training times is the increasing dataset complexity required to reach state-of-the-art performance in real-world applications. To address this challenge, we explore the use of input mixing, where multiple inputs are combined into a single composite input with an associated composite label for training. The goal is for training on the mixed input to achieve a similar effect as training separately on each the constituent inputs that it represents. This results in a lower number of inputs (or mini-batches) to be processed in each epoch, proportionally reducing training time. We find that naive input mixing leads to a considerable drop in learning performance and model accuracy due to interference between the forward/backward propagation of the mixed inputs. We propose two strategies to address this challenge and realize training speedups from input mixing with minimal impact on accuracy. First, we reduce the impact of inter-input interference by exploiting the spatial separation between the features of the constituent inputs in the network's intermediate representations. We also adaptively vary the mixing ratio of constituent inputs based on their loss in previous epochs. Second, we propose heuristics to automatically identify the subset of the training dataset that is subject to mixing in each epoch. Across ResNets of varying depth, MobileNetV2 and two Vision Transformer networks, we obtain upto 1.6 × and 1.8 × speedups in training for the ImageNet and Cifar10 datasets, respectively, on an Nvidia RTX 2080Ti GPU, with negligible loss in classification accuracy.

Dynamics analysis and FPGA implementation of discrete memristive cellular neural network with heterogeneous activation functions

The activation function, as an important component of artificial neural networks, endows neural networks with rich dynamical phenomena by virtue of its nonlinear properties. However, especially for cellular neural networks (CNNs), the exploration of neural networks consisting of heterogeneous activation functions has not been sufficient. In this article, we introduce heterogeneous activation functions in discrete cellular neural network (DCNN) for the first time. In order to enhance the dynamics of the original DCNN, we propose our discrete memristive cellular neural network (DMCNN) based on a memristor with a sinusoidal function. Using various methods of dynamical characterization, such as Lyapunov exponential analysis, bifurcation diagrams and spectral entropy, the rich dynamical behaviour of the proposed model is comprehensively investigated, including hyperchaos, transient chaos, high spectral entropy and multiple types of coexisting attractors in the proposed model. Last but not least, the hardware platform of the proposed model is implemented by field programmable gate array (FPGA), and the dynamics of the proposed model is verified by a combination of software simulation and hardware experiments. The new exploration of DMCNN with heterogeneous activation functions in this article lays the foundation for further research into neural networks with complex dynamical behaviour.

DLAS: A Conceptual Model for Across-Stack Deep Learning Acceleration

Deep Neural Networks (DNNs) are very computationally demanding, which presents a significant barrier to their deployment, especially on resource-constrained devices. Significant work from both the machine learning and computing systems communities has attempted to accelerate DNNs. However, the number of techniques available and the required domain knowledge for their exploration continues to grow, making design space exploration (DSE) increasingly difficult. To unify the perspectives from these two communities, this paper introduces the Deep Learning Acceleration Stack (DLAS), a conceptual model for DNN deployment and acceleration. We adopt a six-layer representation that organizes and illustrates the key areas for DNN acceleration, from machine learning to software and computer architecture. We argue that the DLAS model balances simplicity and expressiveness, assisting practitioners from various domains in tackling co-design acceleration challenges. We demonstrate the interdependence of the DLAS layers, and thus the need for co-design, through an across-stack perturbation study, using a modified tensor compiler to generate experiments for combinations of a few parameters across the DLAS layers. Our perturbation study assesses the impact on inference time and accuracy when varying DLAS parameters across two datasets, seven popular DNN architectures, four compression techniques, three algorithmic primitives (with sparse and dense variants), untuned and auto-scheduled code generation, and four hardware platforms. The study observes significant changes in the relative performance of design choices with the introduction of new DLAS parameters (e.g., the fastest algorithmic primitive varies with the level of quantization). Given the strong evidence for the need for co-design, and the high costs of DSE, DLAS offers a valuable conceptual model for better exploring advanced co-designed accelerated deep learning solutions.

USING THE ARDUINO PLATFORM IN THE RESEARCH OF INTERNAL COMBUSTION ENGINES

Modern measuring systems used in the study of internal combustion engines require significant investments. The universal Arduino platform offers ready-made powerful hardware modules for data collection (shields) and control, which work in a wide range of frequencies and signal amplitudes, provide their analysis and processing, allow to implement equipment management, and have a relatively low cost and free software with standard libraries. These facts testify to the high relevance of the use of the Arduino platform in the study of internal combustion engines. The purpose of this work is to determine the possibilities and features of using the Arduino hardware and software platform in the study of processes occurring in energy power plants with internal combustion engines, early prototyping of their control and diagnostic systems. The paper considers the possibility of hardware and software connection of sensors for measuring basic physical quantities to the Arduino platform: crankshaft rotation frequency (based on induction sensors and Hall sensors), pressure, air and fuel flow, temperatures (resistance sensors and thermocouples). In addition, the characteristics (dependence of the measurement parameter on the output value) of some popular sensors used in the study of internal combustion engines are given. It is important to consider that the equipment used for the study of processes in the ICE must meet the certification requirements. In the case of Arduino, this certification may not be available. However, the recommendations for the use of various sensors, shields and Arduino boards can be applied if the measured quantity is not the determining factor. It is important to consider these limitations and use Arduino with an understanding of the capabilities and limitations of the platform. The use of Arduino in the educational process in the training of specialists in the field of power engineering and in the research of energy installations of vehicles will allow students to apply the knowledge, skills and abilities acquired in the learning process at the hardware and software levels, which will create additional motivation for further study devices and principles of operation of automated and automatic vehicle control systems.

Critical infrastructure monitoring in CBRNe scenarios: a reliable and robust communication network for distributed multimodal sensors

The malicious misuse of CBRNe agents can inflict extensive damage to critical infrastructure and terrorize public society. The effects of such attacks can range from financial or structural drawbacks to significant casualties. Recent terror attacks in Cucuta (2021) or Kabul (2021) demonstrate the need to protect infrastructures such as airports, power plants, and transportation infrastructure. In the face of these modern-day threats, the need for a reliable and effective monitoring system for critical infrastructure has become increasingly important. In a first contribution to the long-term development, we present an initial version of a concept that requires the implementation paradigm of diverse redundancy and secure communication using MQTTS relying on the included cipher suite. An initial set of node types is defined and assigned requirements for implementation. The software tools architecture we propose based on these requirements is designed to support network operators and developers by providing standard features for network management and sensor node implementation. This includes the definition of a secure semi-automatic onboarding process for new sensor nodes, which is presented in detail. We strive for independency from specific hardware platforms, software frameworks, and network technologies to enable an open standard for communication within a critical infrastructure sensor network and also between such networks in the future.

Parametric controllable planar multi-scroll chaotic attractors in a 3-D memristive tabu learning single neuron model

In this paper, a three-dimensional (3D) autonomous tabu learning single neuron model is proposed, which is achieved by using a sinusoidal activation function and introducing a memristor synapse. This model exhibits the remarkable capability to produce a series of planar multi-scroll chaotic attractors, and its unique feature lies in the ability to control the number of scrolls. The investigation of the planar multi-scroll chaotic attractors and its dynamical behaviors is conducted through the analysis of phase plane portraits, bifurcation diagrams, and spectral entropies. The numerical simulations unveil a compelling relationship between the number of chaotic scrolls and specific control parameters governing the model. To further validate the findings, a 3D autonomous tabu learning single neuron model is implemented on a digital hardware platform. In an effort to extend the practical significance of this research, the multi-scroll chaotic phenomenon generated by the proposed model is deployed for image encryption. The fusion of mathematical modeling, digital hardware implementation, and practical application underscores the universality and significance of the proposed single neuron model in the fields of chaotic systems and engineering applications.

Controlling user access with scripting to mitigate cyber-attacks

With the growing threats in the dynamic digital landscape, organizations are grappling with the difficult task of managing cyber-attacks by restricting user accessibility to most communication systems. This study aimed to develop a Linux script as a proactive response to the growing of cyber-attacks in organizational settings to prevent attacking vectors based on user negligence. The primary objective was to devise inventive solutions through script automation to restrict user access to the organizational network within predefined time frames, thereby mitigating potential cyber threats. The criteria used to define predefined time frames for user access, include the restriction of users on off duty times, The scripting seeks to address cyber-attacks targeted such as cross-site scripting (XSS) and SQL injection. To achieve the goal of this paper, the necessary steps and logic required to develop the scripts were taken. For instance, there are several suitable languages such as Bash, Python, or Perl, that can be used to write script. The paper selected Bash because it excels at scripting existing command-line tools and utilities, making it a great choice for automating tasks by using Vim text editor tool. The final scripts were run, some errors were identified, and modified as needed to ensure it worked correctly. This phase was necessary to improve the script's performance, readability, and maintainability by refining code, using efficient algorithms, by incorporating best practices. Finally, the various user inputs and scenarios were deployed to put the script into production for user access management. The adopted script metrics deployed such as Script Execution Time, Resources Utilization, Success Rate and Error Rate helped in evaluating the performance, efficiency, and effectiveness of the scripts. The implementation of the script achieved an executive time of 0.032 s, success rate was 1 and error rate was 0. The practical implementation in organizations can be scaled up or down depending on the organization's size and infrastructure. The compatibility of the developed Linux script with different operating systems, hardware platforms and network architectures will have no impact in adapting to various organizational environments. The script did not show any challenges, but real-world deployment may exhibit some potential challenges and limitations that may be associated with implementing the solution, such as resource constraints and legacy system compatibility.

Research on High-Frequency PGC-EKF Demodulation Technology Based on EOM for Nonlinear Distortion Suppression

In this study, a phase-generated carrier (PGC) demodulation algorithm combined with the extended Kalman filter (EKF) algorithm based on an electro-optic modulator (EOM) is proposed, which can achieve nonlinear distortion (such as modulation depth drift and carrier phase delay) suppression for high-frequency phase carrier modulation. The improved algorithm is implemented on a field-programmable gate array (FPGA) hardware platform. The experimental results by the PGC-EKF method show that total harmonic distortion (THD) decreases from −32.61 to −54.51 dB, and SINAD increases from 32.59 to 47.86 dB, compared to the traditional PGC-Arctan method. This indicates that the PGC-EKF demodulation algorithm proposed in this paper can be widely used in many important fields such as hydrophone, transformer, and ultrasound signal detection.

A high-fidelity lithium-ion battery emulator for electric vehicle application

Lithium-ion (Li-ion) battery has played a key role for the development of electric vehicle (EV) at present, while the Li-ion batteries in the market come from different manufactures. Verifying the performance of the battery management system (BMS) for various battery chemistries is a complex undertaking. This paper proposes a high-fidelity Li-ion battery emulator for EV applications. The emulator utilizes a battery model parameterized by a series of performance tests and a special-designed hardware platform. A three-order battery equivalent circuit model (ECM) is selected to provide the voltage and current reference signal in the processor. Subsequently, the hardware generates the high voltage and current signal in accordance with the reference. To ensure the high accuracy of the battery ECM, a 37 Ah nickel manganese cobalt (NMC) battery was selected for testing under both charge and discharge conditions, as well as across a temperature range of − 30℃ to 45℃. The battery emulator is verified on charge and discharge mode for both accuracy and dynamic performance validations.

Design and analysis of discrete fractional-order chaotic map with offset-boosting behavior

Fractional calculus, as a more accurate tool for depicting the dynamics of complex systems, has been introduced into discrete chaotic maps. To further describe the offset-boosting behavior in discrete systems, a discrete fractional-order chaotic map (DFOCM) based on the Caputo difference operator is constructed. The mapping order of this fractional-order model controls the stability of the fixed point, thereby affecting the dynamic behavior of the map. The dynamics of DFOCM is studied using numerical simulation methods such as bifurcation diagrams and maximum Lyapunov exponents, revealing the presence of multistability. By comparing with integer-order map, it is found that DFOCM exhibit a larger chaotic region. Based on this, the difference between fractional order and integer order offset-boosting behavior is theoretically derived. Specifically, the offset-boosting behavior of fractional-order maps concerning mapping parameters is related to the initial state, which was further demonstrated through numerical simulations. SE complexity proves that the chaotic sequences generated by DFOCM have high unpredictability and pseudo-randomness. Finally, the proposed DFOCM is implemented on the DSP hardware platform, and the physical feasibility of numerical simulation is verified.

A novel method of vehicle height control utilizing semi-active actuator

Inspired by the physical phenomenon that the balance position of the vehicle body changes due to a suspension system with asymmetric damping, this paper presents a novel method to control the vehicle height utilizing semi-active actuators. The proposed vehicle height control method uses the asymmetric damping of the semi-active actuator to control the raising and lowering of the vehicle body, which essentially harnesses the energy from the vehicle’s driving vibrations and has the characteristics of low cost, energy saving and fast response. Aiming at the strong nonlinearity and discontinuity of the piecewise linear nonlinear system of the vehicle body vibration, the homotopy analysis method (HAM) is used to solve the high-order analytical solution, the non-smooth terms are processed through Fourier series expansion, and the iteration method is combined to treat the iteration result as the initial guess solution for the next iteration, improving the convergence accuracy of the series solution and expands the application scope of the HAM. The feasibility of the presented method is investigated from an analytical point of view and the influences of some critical parameters on the responses of the system are analyzed. Finally, the magnetorheological damper (MRD) produced by LORD company is utilized as the actuator, and the vehicle height control experiments system is established by using NI-Labview software and NI-PXI hardware platform, The experiments are performed to verify the feasibility of the proposed vehicle height control method and the system response laws. The proposed method realizes the functions of active actuators using semi-active actuators, and tries a new method and approach for vehicle height control under specific working conditions, such as short-term obstacle surmounting, high-speed steering, emergency avoidance, and rollover prevention.

Optimized k-Nearest neighbors search implementation on resource-constrained FPGA platforms

The k-Nearest Neighbors (kNN) algorithm is a fundamental machine learning classification technique with wide-ranging applications. Among various kNN implementation choices, FPGA-based heterogeneous systems have gained popularity due to FPGA's inherent parallelism, energy efficiency, and reconfigurability. However, implementing the kNN algorithm on resource-constrained embedded FPGA platforms, typically characterized by constrained programmable resources shared among various application-specific hardware units, necessitates a kNN accelerator architecture that balances high performance, hardware efficiency, and flexibility. To address this challenge, in this paper, we present a kNN hardware accelerator unit designed to optimize resource utilization by utilizing sequential, i.e. accumulation-based, instead of pipelined/parallel distance computations. The proposed architecture incorporates two key algorithmic optimizations to reduce the iteration count of the sequential distance computation loop: a dynamic lower bound enabling early termination of the distance computation and an online element selection that maximizes partial distance growth per iteration. We further enhance the accelerator's performance by incorporating multiple optimized sequential distance computation units, each dedicated to processing a segment of the training dataset. Our experiments demonstrate that the proposed approach is scalable, making it applicable to various hardware platforms and resource constraints. In particular, when implemented on an AMD Zynq device, the proposed single-core kNN accelerator occupies a mere 5 % of the FPGA's resources while delivering a speedup of 3 – 5 times compared to the kNN software implementation running on the accompanying ARM A9 processor. For the 8-core kNN accelerator, the resource utilization stands at 30 %, while the speedup factor ranges between 25 and 35.

Software Optimization and Design Methodology for Low Power Computer Vision Systems

This tutorial paper addresses a low power computer vision system as an example of a growing application domain of neural networks, exploring various technologies developed to enhance accuracy within the resource and performance constraints imposed by the hardware platform. Focused on a given hardware platform and network model, software optimization techniques, including pruning, quantization, low-rank approximation, and parallelization, aim to satisfy resource and performance constraints while minimizing accuracy loss. Due to the interdependence of model compression approaches, their systematic application is crucial, as evidenced by winning solutions in the Lower Power Image Recognition Challenge (LPIRC) of 2017 and 2018. Recognizing the typical heterogeneity of processing elements in contemporary hardware platforms, the effective utilization through parallelizing neural networks emerges as increasingly vital for performance enhancement. The paper advocates for a more impactful strategy—designing a network architecture tailored to a specific hardware platform. For detailed information on each technique, the paper provides corresponding references.

The Design and Real-Time Optimization of an EtherCAT Master for Multi-Axis Motion Control

To address the issues of low bandwidth, weak real-time performance, and poor synchronization in traditional fieldbuses for multi-axis motion control, a solution for the implementation of an EtherCAT master based on the IgH EtherCAT Master open-source software framework and an embedded hardware platform is proposed. On a hardware platform centered around the AM64x Sitara processor, a Linux real-time operating system based on the Xenomai real-time kernel is constructed, and the IgH master framework is ported to realize a high-performance EtherCAT master. The configuration process of the EtherCAT bus is detailed, a master application program is developed, and methods for the real-time performance optimization of the master—such as exclusive CPU usage by the master process and the optimization of the network card driver—are proposed. Finally, experiments are conducted on a six-axis servo control platform, with the packet analysis of the periodic EtherCAT data frames sent by the master. The experimental results show that the optimized master, under a high-speed communication cycle of 500 microseconds, maintains maximum jitter within 20 microseconds and average jitter within 1 microsecond, meeting the requirements for high-precision multi-axis motion control.

Quantum advantage and stability to errors in analogue quantum simulators

Several quantum hardware platforms, while being unable to perform fully fault-tolerant quantum computation, can still be operated as analogue quantum simulators for addressing many-body problems. However, due to the presence of errors, it is not clear to what extent those devices can provide us with an advantage with respect to classical computers. In this work, we make progress on this problem for noisy analogue quantum simulators computing physically relevant properties of many-body systems both in equilibrium and undergoing dynamics. We first formulate a system-size independent notion of stability against extensive errors, which we prove for Gaussian fermion models, as well as for a restricted class of spin systems. Remarkably, for the Gaussian fermion models, our analysis shows the stability of critical models which have long-range correlations. Furthermore, we analyze how this stability may lead to a quantum advantage, for the problem of computing the thermodynamic limit of many-body models, in the presence of a constant error rate and without any explicit error correction.

An E-nose system for identification and quantification of hazardous gas mixtures using a combined strategy of CNNs and attentional mechanisms

The chemical industry generates a broad spectrum of hazardous gases, presenting significant challenges for conventional detection methods due to their diverse chemical properties and low concentration levels. E-nose systems, employing sensor arrays, offer significant potential for the determination of gas mixtures. This study presents a novel E-nose algorithm, CNN-ECA, which integrated CNNs and attention mechanisms to improve the recognition accuracy of E-nose systems. By integrating the attention mechanism module into CNN’s convolutional operations, the algorithm emphasizes critical feature information. Three hazardous gases (ammonia, methanol, and acetone) and their mixtures were chosen as target gases. CNNs were combined with various attention mechanism networks (SENet, ECA, and CBAM) to construct models, which were then employed to train and evaluate data collected from the sensor array. The results were compared with traditional network models (KNN, SVM, and CNN). Experimental findings indicated that the prediction performance of CNN models combined with attention mechanism networks surpassed that of traditional network models. Particularly, the CNN-ECA network model demonstrated the highest performance in both qualitative and quantitative analyses. This study presents a promising solution for mixed gas detection by synergizing CNN and attention mechanism networks, thereby enhancing the accuracy and reliability of mixed gas measurements. Moreover, capitalizing on the lightweight architecture of the CNN-ECA model, transfer learning techniques were employed to adapt it for deployment on the Raspberry Pi hardware platform. This facilitates the development of a real-time E-nose system for gas detection.

A new four-dimensional chaotic system with rich transitional characteristics between dissipative and conservative.

The general form of the Hamiltonian function serves as the foundation for the creation of a new four-dimensional chaotic system in this study. We discover that the external excitation parameter d, the internal parameter a, and all initial values have a transforming influence on the system property. Additionally, the corresponding fractional-order chaotic system in accordance with the constructed four-dimensional chaotic system is proposed. It is found that as the order q rises, the system transforms gradually from a dissipative system to a conservative system. Multiple coexisting attraction flows based on the Hamiltonian energy magnitude are present in this dual-property chaotic system. The complexity analysis shows that the system has a high level of complexity. NIST test indicates that the chaotic sequences produced by this dual-property chaotic system exhibit good pseudo-randomness. Finally, a Digital Signal Processing-based hardware platform confirms the physical realizability of the system.

An Open Approach to Autonomous Vehicles

The evolution of autonomous vehicle (AV) technology is rapidly transforming the automotive industry, driving innovations in safety, efficiency, and user experience. This research paper explores the impact of open approaches on the development and deployment of autonomous vehicles. By leveraging open-source software and hardware platforms, such as OpenPilot and Apollo, the study highlights how collaborative, transparent methodologies foster innovation and accelerate technological advancements. The paper examines the technical, regulatory, and ethical challenges associated with open-source AV development, including data management, sensor integration, and compliance with safety standards. Through a comprehensive analysis of successful open-source projects and emerging trends, the research identifies the benefits of open approaches in promoting cross-disciplinary collaboration and addressing complex challenges. The findings underscore the potential of open-source solutions to drive the future of autonomous vehicle technology, offering insights into their role in shaping industry standards and regulatory frameworks

Advanced computer architecture optimization for machine learning/deep learning

Abstract The recent progress in Machine Learning (Géron, 2022) and particularly Deep Learning (Goodfellow, 2016) models exposed the limitations of traditional computer architectures. Modern algorithms demonstrate highly increased computational demands and data requirements that most existing architectures cannot handle efficiently. These demands result in training speed, inference latency, and power consumption bottlenecks, which is why advanced methods of computer architecture optimization are required to enable the development of ML/DL-dedicated efficient hardware platforms (Engineers, 2019). The optimization of computer architecture for applications of ML/DL becomes critical, due to the tremendous demand for efficient execution of complex computations by Neural Networks (Goodfellow, 2016). This paper reviewed the numerous approaches and methods utilized to optimize computer architecture for ML/DL workloads. The following sections contain substantial discussion concerning the hardware-level optimizations, enhancements of traditional software frameworks and their unique versions, and innovative explorations of architectures. In particular, we discussed hardware including specialized accelerators, which can improve the performance and efficiency of a computation system using various techniques, specifically describing accelerators like CPUs (multicore) (Hennessy, 2017), GPUs (Hwu, 2015) and TPUs (Contributors, 2017), parallelism in multicore architectures, data movement in hardware systems, especially techniques such as caching and sparsity, compression, and quantization, other special techniques and configurations, such as using specialized data formats, and measurement sparsity. Moreover, this paper provided a comprehensive analysis of current trends in software frameworks, Data Movement optimization strategies (A.Bienz, 2021), sparsity, quantization and compression methods, using ML for architecture exploration, and, DVFS (Hennessy, 2017),, which provides strategies for maximizing hardware utilization and power consumption during training, machine learning, dynamic voltage, and frequency scaling, runtime systems. Finally, the paper discussed research opportunity directions and the possibilities of computer architecture optimization influence in various industrial and academic areas of ML/DL technologies. The objective of implementing these optimization techniques is to largely minimize the current gap between the computational needs of ML/DL algorithms and the current hardware’s capability. This will lead to significant improvements in training times, enable real-time inference for various applications, and ultimately unlock the full potential of cutting-edge machine learning algorithms.

Design and Application of an Onboard Particle Identification Platform Based on Convolutional Neural Networks

Space radiation particle detection plays a crucial role in scientific research and engineering practice, especially in particle species identification. Currently, commonly used in-orbit particle identification techniques include telescope methods, electrostatic analysis time of flight (ESA × TOF), time-of-flight energy (TOF × E), and pulse shape discrimination (PSD). However, these methods usually fail to utilize the full waveform information containing rich features, and their particle identification results may be affected by the random rise and fall of particle deposition and noise interference. In this study, a low-latency and lightweight onboard FPGA real-time particle identification platform based on full waveform information was developed utilizing the superior target classification, robustness, and generalization capabilities of convolutional neural networks (CNNs). The platform constructs diversified input datasets based on the physical features of waveforms and uses Optuna and Pytorch software architectures for model training. The hardware platform is responsible for the real-time inference of waveform data and the dynamic expansion of the dataset. The platform was utilized for deep learning training and the testing of the historical waveform data of neutron and gamma rays, and the inference time of a single waveform takes 4.9 microseconds, with an accuracy rate of over 97%. The classification expectation FOM (figure-of-merit) value of this CNN model is 133, which is better than the traditional pulse shape discrimination (PSD) algorithm’s FOM value of 0.8. The development of this platform not only improves the accuracy and efficiency of space particle discrimination but also provides an advanced tool for future space environment monitoring, which is of great value for engineering applications.