Hardware Optimization Research Articles

Comparison Distributed and Parallel Machine Learning: Evidence from Models, Principles and Application Scenarios

Abstract. Contemporarily, the demand for processing large-scale data has been rapidly increasing, prompting continuous advancements in the field of machine learning. This study examines the state of parallel and distributed machine learning at present, both of which aim to enhance computational efficiency when dealing with large datasets and demonstrate promising applications across various domains. As the scale of data escalates, traditional single-machine learning methods are becoming increasingly inadequate, which lead to the emergence of parallel and distributed machine learning. These approaches enable substantial computations to be performed more efficiently through the collaboration of multi-core CPUs or multiple computing nodes. This research conducts an in-depth analysis of the inherent challenges associated with these methods, including data transmission latency, synchronization requirements, and user privacy concerns. Ultimately, this research emphasizes the tremendous potential of both methods for future applications, a potential that is bolstered by ongoing advancements in hardware and algorithm optimization. These results provide valuable insights for practitioners in the field and offers guidance for future research directions.

Benchmarking Mobile Deep Learning Software

Deploying deep learning (DL) on mobile devices has become increasingly prevalent. DL software libraries are crucial for efficient on-device inference, alongside algorithms and hardware. However, there has been limited understanding on the performance of modern DL libraries. We fill this gap by benchmarking 6 popular DL libraries and 15 diverse models across 10 mobile devices, which reveal an unsatisfactory landscape of mobile DL: their performance is highly disparate and fragmented across different models and hardware, and the impacts often surpass algorithm or hardware optimizations, such as model quantization and GPU/NPU-based computing. Finally, we provide practical implications for stakeholders in the DL library ecosystem, and envision a more ambitious picture of future mobile AI landscape in the LLM era.

Accelerating Neural Network Inference in Handwritten Digit Recognition — Comparative Study

In deep learning, one of the popular subclasses of multilayer perceptron is the Convolutional Neural Networks (CNNs), optimized for image and semantic segmentation tasks. With a large number of floating-point operations and relatively less data transfer in the training phase, CNNs are well suited to be handled by parallel architectures. The high accuracy of CNNs comes at the cost of consequential compute and memory demands. In this article, we present a comparative analysis of a pre-trained CNN framework for recognizing a handwritten digit on different processing platforms. The performance of the neural network as the function of the parallel processing platforms offers parametric insights and factors that influence the inference on state-of-the-art Graphics Processing Units (GPUs) and systolic arrays such as Eyeriss and Tensor Processing Unit (TPU). Through inference time analysis, we observed that the systolic arrays can outperform upto 58.7 times better than a Turing architecture powered GPU. We show that while the efficiency with which the available resources are utilized is higher in the existing GPUs (upto 32%) than the TPUs, the efficiency of application-specific systolic arrays can be on par with that of GPUs. We present the results obtained from three different customized systolic array-based platforms which can be adopted by the designers to decide the hardware optimization goal.

Creative and accurate method for optimal hardware implementation of neurons and biological cells: Application in FPGA-based implementation of cardiac pacemaker cell

The sinoatrial (SA) node cells play a vital role as the principal pacemaker in mammalian hearts, generating regular and spontaneous action potentials to regulate the heart’s rhythm. Comprehending the intricate activity of the SA node’s operation necessitates a collection of differential formulas that tackle non-linear functions. The study presents a new technique to improve the digital representation of the SA node cell model, offering benefits such as decreased hardware needs, enhanced processing speed and accuracy, and reduced implementation expenses by transforming the original model’s differential equations into a unified trigonometric function. This transformation significantly simplifies the computational complexity by eliminating the need for multipliers, resulting in a streamlined set of mathematical expressions. The digital implementation of this novel method can be efficiently realized using the Coordinate Rotation Digital Computer (CORDIC) algorithm, which circumvents the necessity for cumbersome mathematical operations. To demonstrate the viability of this approach, the proposed model is successfully synthesized and implemented on a Field-Programmable Gate Array (FPGA). The results of the implementation demonstrate a significant rise in the operating frequency, which is approximately 6.14 times greater than that of the original model. Furthermore, there is a notable 45 percent decrease in power usage. The lowered hardware needs make significant scalability possible, thus allowing for the inclusion of approximately 12 times as many SA node cells on a sole FPGA board in comparison to the original design.

Hardware Acceleration for High-Volume Operations of CRYSTALS-Kyber and CRYSTALS-Dilithium

Many high-demand digital services need to perform several cryptographic operations, such as key exchange or security credentialing, in a concise amount of time. In turn, the security of some of these cryptographic schemes is threatened by advances in quantum computing, as quantum computer could break their security in the near future. Post-quantum cryptography (PQC) is an emerging field that studies cryptographic algorithms that resist such attacks. The National Institute of Standards and Technology (NIST) has selected the CRYSTALS-Kyber Key Encapsulation Mechanism and the CRYSTALS-Dilithium Digital Signature algorithm as primary PQC standards. In this article, we present field-programmable gate array (FPGA)-based hardware accelerators for high-volume operations of both schemes. We apply high-level synthesis (HLS) for hardware optimization, leveraging a batch processing approach to maximize the memory throughput and applying custom HLS logic to specific algorithmic components. Using reconfigurable FPGAs, we show that our hardware accelerators achieve speedups between 3 \(\times\) and 9 \(\times\) over software baseline implementations, even over ones leveraging CPU vector architectures. Furthermore, the methods used in this study can also be extended to the new CRYSTALS-based NIST FIPS drafts, ML-KEM and ML-DSA, with similar acceleration results.

Hardware optimization for effective switching power reduction during data compression in GOLOMB rice coding.

Loss-less data compression becomes the need of the hour for effective data compression and computation in VLSI test vector generation and testing in addition to hardware AI/ML computations. Golomb code is one of the effective technique for lossless data compression and it becomes valid only when the divisor can be expressed as power of two. This work aims to increase compression ratio by further encoding the unary part of the Golomb Rice (GR) code so as to decrease the amount of bits used, it mainly focuses on optimizing the hardware for encoding side. The algorithm was developed and coded in Verilog and simulated using Modelsim. This code was then synthesised in Cadence Encounter RTL Synthesiser. The modifications carried out show around 6% to 19% reduction in bits used for a linearly distributed data set. Worst-case delays have been reduced by 3% to 8%. Area reduction varies from 22% to 36% for different methods. Simulation for Power consumption shows nearly 7% reduction in switching power. This ideally suggest the usage of Golomb Rice coding technique for test vector compression and data computation for multiple data types, which should ideally have a geometrical distribution.

Introductory Tutorials for Simulating Protein Dynamics with GROMACS.

Atomistic molecular dynamics (MD) simulations have become an indispensable tool for investigating the structure, dynamics, and energetics of biomolecules. Continual optimization of software algorithms and hardware has enabled investigators to access biologically relevant time scales in feasible amounts of computing time. Given the widespread use and utility of MD simulations, there is considerable interest in learning essential skills in performing them. Here, we present a set of introductory tutorials for performing MD simulations of proteins in the popular, open-source GROMACS package. Three exercises are detailed, including simulating a single protein, setting up a protein complex, and performing umbrella sampling simulations to model the unfolding of a short polypeptide. Essential features and input settings are illustrated throughout. The purpose of these tutorials is to provide new users with a general understanding of foundational workflows, from which they can design their own simulations.

Evaporated Copper‐Based Perovskite Dynamic Memristors for Reservoir Computing Systems

AbstractDynamic memristors are considered as the optimal hardware devices for reservoir computing (RC) enabled by their nonlinear conductance variations. This significantly reduces the extensive training workload typically required by traditional neural networks. Lead halide perovskites, with their tunable band structure and active ion migration properties, have emerged as highly promising materials for developing dynamic memristors. However, large‐scale and consistently stable production remains a challenge for perovskite functional films, while lead elements' toxicity and environmental impact also partly restrict their practical device utilization. In this work, lead‐free copper‐based perovskite (i.e., CsCu2I3) films are prepared by thermal evaporation for constructing dynamic memristors. The effective conductivity modulation of CsCu2I3‐based memristor can be utilized in artificial neural networks, achieving a high handwritten digit recognition accuracy of 91.2%. In addition, the RC system is also constructed based on the dynamic behavior of the devices, by which a letter recognition accuracy of 98.2% with simple training is achieved. This technology provides a feasible pathway to construct copper‐based perovskite dynamic memristors for future neural network information processing.

Энергоэффективность многоядерных процессоров: сравнительное исследование аппаратных технологий

Background: The constant increase in processing demand has prompted the development of multi-core processors, which are critical to obtaining high performance in a variety of computing devices. However, this growth in processing capacity comes at the expense of increased energy consumption, posing substantial problems for energy efficiency. Objective: The article will examine and compare several hardware solutions for improving energy efficiency in multi-core processors, offering a full overview of their effectiveness and application. Methods: We used a quantitative research methodology, simulating multi-core processing environments with various hardware optimization techniques such as Dynamic Voltage and Frequency Scaling (DVFS), Clock Gating, and Power Gating. Each technique was evaluated in a variety of operational circumstances to establish its effect on energy consumption and processing efficiency. Results: The findings show that DVFS provides significant energy savings with negligible performance trade-offs in cases with moderate workloads. In contrast, Clock Gating performed best in low workload settings, whereas Power Gating performed best in high-load conditions, dramatically reducing idle power consumption. Conclusion: The comparison research demonstrates that no single technique beats all others; rather, the choice of an effective energy efficiency strategy is significantly influenced by unique workload factors. A hybrid strategy, which combines both techniques depending on real-time workload demands, can greatly improve the energy efficiency of multi-core processors. This study advances our understanding of energy management in advanced computing systems, providing insights for future research and practical applications in the field of energy-efficient computing.

A Novel Method for Optimising Energy Efficiency in High-Performance Computing Systems

This paper introduces a new method to promote energy efficiency in computing systems. The rapidly growing demand for computational power in high-performance computing (HPC) systems is accompanied by a significant increase in energy consumption. This research investigates the many ways to minimise energy consumption in HPC systems without sacrificing computational performance. Building upon previous research we combined and refined ideas to develop an optimised approach. Our method utilises a tri-modular framework incorporating an Energy-Efficient Hardware Design, Resource Management, and Optimisation, and Server Virtualisation. The first module employs a method of designing smart energy-efficient hardware. This method incorporates leakage reduction techniques such as clock gating and sleep states. The second module uses a technique focused on optimising server software. This method is based on Dynamic Voltage and frequency scaling (DVFS) for power management in data centres. The third module is based on server virtualisation and employs software to create multiple virtual machines on a single physical server allowing for a significant reduction in energy and hardware costs. The methods used in these three modules are systematically integrated to produce a more efficient consumption of electricity. This will allow for the computing system to minimise energy consumption without compromising computational power.

Systematic Optimization of Electromagnet Hardware for Electromagnetic Suspension: A Fusion of Simulation and Multi-objective Optimization Techniques

Systematic Optimization of Electromagnet Hardware for Electromagnetic Suspension: A Fusion of Simulation and Multi-objective Optimization Techniques

Design and application of a high-precision counterweighted self-calibrating surface thermometer.

In this study, a high-precision counterweight self-calibrating surface thermometer is designed to reduce human and environmental influences on a thermocouple surface thermometer during measuring. A self-weighted spring structure based on a copper substrate is designed to ensure perfect contact between the surface thermometer and the temperature source. In conjunction, a wind guard is coupled with insulating materials to optimize the thermal exchange of the surface thermometer. Subsequently, the maximum error is reduced to ±1.5 °C by system hardware optimization. However, hardware calibration alone is insufficient. Furthermore, a back propagation neural network is employed to calibrate the surface thermometer. Temperature sensor data are collected under various surface source temperatures and airflow velocities to train the neural network. Hence, the effectiveness of the proposed Gaussian function in enhancing the measurement accuracy of the surface temperature sensor is demonstrated. The results show higher stability and repeatability in temperature measurement than thermocouple-based surface thermometers. The proposed thermometer exhibits robustness against environmental and operational variability with a maximum indication error of -0.2 °C. In contrast, the maximum error of the surface thermometer is between -2.8 and -6.8 °C. Regarding repeatability, the standard deviation with the proposed device is 0.2%, highlighting its accuracy and consistency of performance. These results can mostly be attributed to the synergistic effect of clever mechanical design and software optimization, resulting in a surface thermometer with outstanding accuracy and repeatability.

Evaluation of a standard analytical model and definition of efficiency metrics for lunar and planetary excavation

To sustainably live on and explore the lunar and other planetary surfaces, in situ regolith will need to be used for resource extraction and infrastructure development. To evaluate and ensure such safety and efficiency of excavation operations in variable, site-specific regolith terranes with given equipment designs, reliable computational models are necessary. The work here evaluates the accuracy and reliability of a standard, well-established excavation force prediction model using published data from a previous planetary excavation experimental study and establishes efficiency metrics for force reduced excavation as a proof-of-concept. It is shown that the established excavation force prediction model is not able to simulate planetary excavation mechanics sufficiently and hence more advanced models of lunar and planetary excavation are needed to enable safer, faster, and more economical resource acquisition hardware development. The case study of the efficiency metrics being applied to the published data shows informative trends that can be used in future excavation hardware and tool path optimizations. These are recommended for use in future planetary excavation modeling and planning workflows.

Going beyond hardware limitations with advanced stray light calibration for the Metop-3MI space instrument

Space optical instruments play a pivotal role in enhancing our understanding of the universe and our planet, and are crucial in addressing the urgent challenges posed by climate change. In this context, stray light has emerged as a primary performance limitation. Originating from ghost reflections or scattering, it obscures essential details and introduces false information into images. With the demand for increasingly high-performing instruments, mitigation through hardware optimization is becoming insufficient. We are entering an era where future instruments require a stray light correction algorithm to meet user specifications, necessitating extensive on-ground calibration. This paper examines the Metop-3MI Earth observation instrument, which, with wide field of view, broad spectral range, and multi-polarization capabilities, epitomizes the challenges of stray light calibration and correction. A custom calibration apparatus was constructed to evaluate the complex stray light dependence on field-of-view, wavelength, and polarization. Data were processed, and stray light kernels database was derived, which then fed into a specially developed correction algorithm. Applied to the image of an extended scene, it effectively reduces stray light by a remarkable factor of 91. This achievement sets a new standard for low-stray-light instruments and provides a comprehensive case study for future missions.

On the emerging potential of quantum annealing hardware for combinatorial optimization

Over the past decade, the usefulness of quantum annealing hardware for combinatorial optimization has been the subject of much debate. Thus far, experimental benchmarking studies have indicated that quantum annealing hardware does not provide an irrefutable performance gain over state-of-the-art optimization methods. However, as this hardware continues to evolve, each new iteration brings improved performance and warrants further benchmarking. To that end, this work conducts an optimization performance assessment of D-Wave Systems’ Advantage Performance Update computer, which can natively solve sparse unconstrained quadratic optimization problems with over 5,000 binary decision variables and 40,000 quadratic terms. We demonstrate that classes of contrived problems exist where this quantum annealer can provide run time benefits over a collection of established classical solution methods that represent the current state-of-the-art for benchmarking quantum annealing hardware. Although this work does not present strong evidence of an irrefutable performance benefit for this emerging optimization technology, it does exhibit encouraging progress, signaling the potential impacts on practical optimization tasks in the future.

Optimizing Printhead Design for Enhanced Temperature Control in Extrusion-Based Bioprinting.

This study addresses critical challenges in the field of tissue engineering, specifically in the optimization of bioprinting technologies for the construction of complex, multicellular tissues. By utilizing a homemade piston-driven extrusion-based bioprinting (EBB) printhead, we performed detailed thermal and flow analyses to investigate the effects of temperature variations on the extrusion process of temperature-sensitive gelatin-alginate bioink. Through finite element method (FEM) simulations, we explored the temperature distribution within the printhead and its impact on bioink properties, such as viscosity, pressure, and shear stress. Key findings reveal significant temperature gradients from the printhead barrel to the nozzle tip, influencing bioink extrusion and filament morphology. This study further introduces an innovative hardware optimization with thermal insulators, designed to mitigate heat loss at the nozzle tip and ensure uniform temperature distribution. Both simulation and empirical printing experiments confirm the efficacy of thermal insulators in enhancing bioprinting fidelity and efficiency. This research contributes to the advancement of bioprinting technology by optimizing printhead design, with implications for improving the quality of bioprinted tissues and organs.

Measuring prairie snow water equivalent with combined UAV-borne gamma spectrometry and lidar

Abstract. Despite decades of effort, there remains an inability to measure snow water equivalent (SWE) at high spatial resolutions using remote sensing. Passive gamma ray spectrometry is one of the only well-established methods to reliably remotely sense SWE, but airborne applications to date have been limited to observing kilometre-scale areal averages. Noting the increasing capabilities of unoccupied aerial vehicles (UAVs) and miniaturization of passive gamma ray spectrometers, this study tested the ability of a UAV-borne gamma spectrometer and concomitant UAV-borne lidar to quantify the spatial variability of SWE at high spatial resolutions. Gamma and lidar observations from a UAV (UAV-gamma and UAV-lidar) were collected over two seasons from shallow, wind-blown, prairie snowpacks in Saskatchewan, Canada, with validation data collected from manual snow depth and density observations. A fine-resolution (0.25 m) reference dataset of SWE, to test UAV-gamma methods, was developed from UAV-lidar snow depth and snow survey snow density observations. The ability of UAV-gamma to resolve the areal average and spatial variability of SWE was promising with appropriate flight characteristics. Survey flights flown at a velocity of 5 m s−1, altitude of 15 m, and line spacing of 15 m were unable to capture the average or spatial variability of SWE within the uncertainty of the reference dataset. Slower, lower, and denser flight lines at a velocity of 4 m s−1, altitude of 8 m, and line spacing of 8 m were able to successfully observe areal average SWE and its variability at spatial resolutions greater than 22.5 m. Using a combination of UAV-based gamma SWE and UAV-based lidar snow depth improved the spatial representation of SWE substantially and permitted estimation of SWE at a spatial resolution 0.25 m with a ± 14.3 mm error relative to the reference SWE dataset. UAV-borne gamma spectrometry to estimate SWE is a promising and novel technique that has the potential to improve the measurement of shallow prairie snowpacks, and when combined with UAV-borne lidar snow depths, can provide fine-resolution, high-accuracy estimates of prairie SWE. Research on optimal hardware, data processing, and interpolation techniques is called for to further improve this remote sensing product and explore its application in other environments.

Exploring the horizon of AI development: Navigating constraints of chips and power in the technological landscape

With the proliferation of AI technology, machine learning has emerged as a cornerstone of AI systems, facilitating pattern recognition and decision-making through robust data analysis. This encompasses various learning paradigms such as supervised, unsupervised, and reinforcement learning, all of which are indispensable for the advancement of artificial intelligence. Nevertheless, the development of AI necessitates substantial computational resources, with specialized chips serving as the linchpin, particularly in demanding tasks such as deep learning. Dedicated chip development, exemplified by GPUs and TPUs, plays a pivotal role in enhancing the performance of AI systems, notwithstanding challenges related to costs and market monopolies. Moreover, AI systems require significant power support, especially during the training of large-scale models. To address these challenges, this paper reviews the existing literature on modeling techniques aimed at enhancing the efficiency of machine learning and reducing energy consumption. This review encompasses optimal algorithm design, hardware optimization, and spatial modeling. Through the implementation of these approaches, the challenges posed by resource constraints in machine learning scenarios can be effectively mitigated, thereby fostering the continued development and application of AI technology.

The eye of artificial intelligence - Convolutional Neural Networks

Inspired by the biological visual system, the convolutional neural network has been widely studied and invented in the field of artificial intelligence. As one of the important algorithms in artificial neural networks, convolutional neural networks have shown outstanding application potential in fields such as image recognition, computer vision, and natural language processing. This article will focus on exploring the powerful capabilities of convolutional neural networks in image processing. By delving into the implementation process of a convolutional neural network, readers will gain a deeper understanding of its working principles. In addition, this article will briefly introduce three classic models of convolutional neural networks, providing readers with more background knowledge. Next, this paper will analyze in detail two typical application cases of convolutional neural networks in the field of image processing: intelligent transportation systems and dental imaging technology. These cases demonstrate the successful application of convolutional neural networks in practical scenarios, pointing the way for their future development. In the future, convolutional neural networks will be more widely used in fields such as image and video processing as data scale increases and computing power improves. By using techniques such as model compression and hardware optimization, it is made more suitable for low-power and high-efficiency environments, and its interpretability and applicability are enhanced through data augmentation and model interpretation techniques.

SpChar: Characterizing the sparse puzzle via decision trees

Sparse matrix computation is crucial in various modern applications, including large-scale graph analytics, deep learning, and recommender systems. The performance of sparse kernels varies greatly depending on the structure of the input matrix, making it difficult to gain a comprehensive understanding of sparse computation and its relationship to inputs, algorithms, and target machine architecture. Despite extensive research on certain sparse kernels, such as Sparse Matrix-Vector Multiplication (SpMV), the overall family of sparse algorithms has yet to be investigated as a whole. This paper introduces SpChar, a workload characterization methodology for general sparse computation. SpChar employs tree-based models to identify the most relevant hardware and input characteristics, starting from hardware and input-related metrics gathered from Performance Monitoring Counters (PMCs) and matrices. Our analysis enables the creation of a characterization loop that facilitates the optimization of sparse computation by mapping the impact of architectural features to inputs and algorithmic choices. We apply SpChar to more than 600 matrices from the SuiteSparse Matrix collection and three state-of-the-art Arm Central Processing Units (CPUs) to determine the critical hardware and software characteristics that affect sparse computation. In our analysis, we determine that the biggest limiting factors for high-performance sparse computation are (1) the latency of the memory system, (2) the pipeline flush overhead resulting from branch misprediction, and (3) the poor reuse of cached elements. Additionally, we propose software and hardware optimizations that designers can implement to create a platform suitable for sparse computation. We then investigate these optimizations using the gem5 simulator to achieve a significant speedup of up to 2.63× compared to a CPU where the optimizations are not applied.