Floating Point Precision Research Articles

Automated Detection and Repair of Floating-point Precision Problems in Convolutional Neural Network Operators

Convolutional Neural Network (CNN) operators, mostly based on mathematical linear computations, are of vital importance to developing CNN-based software. Existing studies reveal that these operators are prone to floating-point precision problems (FPPs). In a CNN-based application, such problems can be propagated and result in catastrophic consequences. Thus, it is highly desired to detect and repair the FPPs in CNN operators. Considering the FPPs in CNN operators are mainly caused by accumulated floating-point errors and diverse floating-point tensors instead of wrong codes or bad implementations, it requires much time cost and is difficult to tackle these FPPs. In this paper, we propose the first method for the automated detection and repair of FPPs in CNN operators from the perspective of floating-point tensors. To generate diverse tensors with floating-point numbers, we design two levels of mutation rules, namely computation-level mutation and input-level mutation, containing a total of five mutation methods. To detect the FPPs caused by the accumulated floating-point errors, our method uses a weight matrix to guide the progressive mutation. To repair the detected FPPs, our method transforms the error-prone floating-point tensors based on the mathematical rewriting of the floating-point linear computational properties without destroying the original computation. Experimental results show that our methods can detect and repair FPPs in CNN operators effectively and efficiently and could reduce 93.32% to 100% of the FPPs in CNN operators. We conduct a case study on six different widely-used CNN models and confirm that the proposed FPP method is generalizable and effective across a variety of tasks and architectures. Our detection and repair method offers an intuitive way to handle FPPs during development, allowing users to continue building and fine-tuning their models without being slowed down by numerical precision errors. We believe that our method could open up a new way to enhance the quality of CNN operators and CNN-based software.

Quantized neural network for complex hologram generation

Computer-generated holography (CGH) is a promising technology for augmented reality displays, such as head-mounted or head-up displays. However, its high computational demand makes it impractical for implementation. Recent efforts to integrate neural networks into CGH have successfully accelerated computing speed, demonstrating the potential to overcome the trade-off between computational cost and image quality. Nevertheless, deploying neural-network-based CGH algorithms on computationally limited embedded systems requires more efficient models with lower computational cost, memory footprint, and power consumption. In this study, we developed a lightweight model for complex hologram generation by introducing neural network quantization. Specifically, we built a model based on tensor holography and quantized it from 32-bit floating-point precision (FP32) to 8-bit integer precision (INT8). Our performance evaluation shows that the proposed INT8 model achieves hologram quality comparable to that of the FP32 model while reducing the model size by approximately 70% and increasing the speed fourfold. Additionally, we implemented the INT8 model on a system-on-module to demonstrate its deployability on embedded platforms and high power efficiency.

Algorithm 1048: A C++ Class for Robust Linear Barycentric Rational Interpolation

Barycentric rational interpolation is a recent interpolation method with several favourable properties. In this article, we present the BRI class, which features a new C++ class template that contains all variables and functions related to linear barycentric rational interpolation. While several methods exist to evaluate a barycentric rational interpolant, the class is designed to autonomously select the best method to use on a case-by-case basis, as it takes into account the latest results regarding the efficiency and numerical stability of barycentric rational interpolation [ 15 ]. Moreover, we describe a new technique that makes the code robust and less prone to overflow and underflow errors. In addition to the standard C++ data types, the BRI template variables can also be defined with arbitrary precision because the BRI class is compatible with the Multiple Precision Floating-Point Reliable (MPFR) library [ 14 ].

DESIGN AND PERFORMANCE ANALYSIS OF TERNARY LOGIC BASED ALU USING DOUBLE PRECISION FLOATING POINT

DESIGN AND PERFORMANCE ANALYSIS OF TERNARY LOGIC BASED ALU USING DOUBLE PRECISION FLOATING POINT

Single molecule dynamics in a virtual cell combining a 3-dimensional matrix model with random walks

Recent advances in light microscopy have enabled single molecules to be imaged and tracked within living cells and this approach is impacting our understanding of cell biology. Computer modeling and simulation are important adjuncts to the experimental cycle since they aid interpretation of experimental results and help refine, test and generate hypotheses. Object-oriented computer modeling is particularly well-suited for simulating random, thermal, movements of individual molecules as they interact with other molecules and subcellular structures, but current models are often limited to idealized systems consisting of unit volumes or planar surfaces. Here, a simulation tool is described that combines a 3-dimensional, voxelated, representation of the cell consisting of subcellular structures (e.g. nucleus, endoplasmic reticulum, cytoskeleton, vesicles, and filopodia) combined with numerical floating-point precision simulation of thousands of individual molecules moving and interacting within the 3-dimensional space. Simulations produce realistic time-series video sequences comprising single fluorophore intensities and realistic background noise which can be directly compared to experimental fluorescence video microscopy data sets.

High-resolution in vivo 4D-OCT fish-eye imaging using 3D-UNet with multi-level residue decoder.

Optical coherence tomography (OCT) allows high-resolution volumetric imaging of biological tissues in vivo. However, 3D-image acquisition often suffers from motion artifacts due to slow frame rates and involuntary and physiological movements of living tissue. To solve these issues, we implement a real-time 4D-OCT system capable of reconstructing near-distortion-free volumetric images based on a deep learning-based reconstruction algorithm. The system initially collects undersampled volumetric images at a high speed and then upsamples the images in real-time by a convolutional neural network (CNN) that generates high-frequency features using a deep learning algorithm. We compare and analyze both dual-2D- and 3D-UNet-based networks for the OCT 3D high-resolution image reconstruction. We refine the network architecture by incorporating multi-level information to accelerate convergence and improve accuracy. The network is optimized by utilizing the 16-bit floating-point precision for network parameters to conserve GPU memory and enhance efficiency. The result shows that the refined and optimized 3D-network is capable of retrieving the tissue structure more precisely and enable real-time 4D-OCT imaging at a rate greater than 10 Hz with a root mean square error (RMSE) of ∼0.03.

Fast and robust analog in-memory deep neural network training

Analog in-memory computing is a promising future technology for efficiently accelerating deep learning networks. While using in-memory computing to accelerate the inference phase has been studied extensively, accelerating the training phase has received less attention, despite its arguably much larger compute demand to accelerate. While some analog in-memory training algorithms have been suggested, they either invoke significant amount of auxiliary digital compute—accumulating the gradient in digital floating point precision, limiting the potential speed-up—or suffer from the need for near perfectly programming reference conductance values to establish an algorithmic zero point. Here, we propose two improved algorithms for in-memory training, that retain the same fast runtime complexity while resolving the requirement of a precise zero point. We further investigate the limits of the algorithms in terms of conductance noise, symmetry, retention, and endurance which narrow down possible device material choices adequate for fast and robust in-memory deep neural network training.

Reduced floating-point precision in regional climate simulations: an ensemble-based statistical verification

Abstract. The use of single precision in floating-point representation has become increasingly common in operational weather prediction. Meanwhile, climate simulations are still typically run in double precision. The reasons for this are likely manifold and range from concerns about compliance and conservation laws to the unknown effect of single precision on slow processes or simply the less frequent opportunity and higher computational costs of validation. Using an ensemble-based statistical methodology, Zeman and Schär (2022) could detect differences between double- and single-precision simulations from the regional weather and climate model COSMO. However, these differences are minimal and often only detectable during the first few hours or days of the simulation. To evaluate whether these differences are relevant for regional climate simulations, we have conducted 10-year-long ensemble simulations over the European domain of the Coordinated Regional Climate Downscaling Experiment (EURO-CORDEX) in single and double precision with 100 ensemble members. By applying the statistical testing at a grid-cell level for 47 output variables every 12 or 24 h, we only detected a marginally increased rejection rate for the single-precision climate simulations compared to the double-precision reference based on the differences in distribution for all tested variables. This increase in the rejection rate is much smaller than that arising from minor variations of the horizontal diffusion coefficient in the model. Therefore, we deem it negligible as it is masked by model uncertainty. To our knowledge, this study represents the most comprehensive analysis so far on the effects of reduced precision in a climate simulation for a realistic setting, namely with a fully fledged regional climate model in a configuration that has already been used for climate change impact and adaptation studies. The ensemble-based verification of model output at a grid-cell level and high temporal resolution is very sensitive and suitable for verifying climate models. Furthermore, the verification methodology is model-agnostic, meaning it can be applied to any model. Our findings encourage exploiting the reduction of computational costs (∼30 % for COSMO) obtained from reduced precision for regional climate simulations.

Intercomparison to validate implementations of different laboratories of algorithms for the analysis of surface texture

The surface texture is an important feature of machine parts for any kind of contact. The quantification of its properties has been a well-established area in quality assurance for many decades. With the increasing demand for precision and interchangeability in the globalised marketplace, the results of surface analysis must be unambiguous and reliable. This is why the International Standardization’s Technical Committee on Geometric Product Specification has updated its standards. Consequently, instrument manufacturers and calibration laboratories need to ensure that their analysis software complies with the new ISO standards. An intercomparison has therefore been carried out to validate surface texture analysis software. Over the past two decades, a number of tests have been conducted on surface texture measurement and analysis methods, including issues of instrument response, discretisation and quantisation. This article examines the numerical and computational aspects of analysis algorithms that process data sets of discrete values of finite floating-point precision as input. The method of testing texture analysis algorithms utilising software measurement standards (softgauges) and the specification of the test tasks are explained. The results of the comparison are presented and discussed.

Batched sparse and mixed-precision linear algebra interface for efficient use of GPU hardware accelerators in scientific applications

Batched Sparse Linear Algebra has become an emergent processing mode on modern hardware accelerators based on Graphics Processing Units (GPUs) developed over the years to serve as the main compute devices in the largest computing clusters and supercomputers. We propose a set solver interface designs for batched sparse numerical solvers on these hardware accelerators. We motivate our specific designs by both their use in scientific applications of national importance and also by the possibility of implementing them in an efficient and portable manner with multiple options for vendor-specific optimizations. We present the C language interface calls for the linker-agnostic interchange of functional entry points. We also show how using C++ for the batched solvers simplifies the interface design while giving the user much broader set of opportunities for customization, testing, and debugging. We also cover in our proposals the option of exploiting multiple floating-point arithmetic precisions to directly match the application needs in terms of accuracy. Finally, a selected sample of performance experiments show how our proposed interface can be efficiently implemented to outperform the available alternatives many times over. In the end, we plan for an ongoing evolution of our newly proposed interface standard to keep up with the updates in programming languages, accelerator hardware, and application needs.

Quantized non-volatile nanomagnetic domain wall synapse based autoencoder for efficient unsupervised network anomaly detection

Anomaly detection in real-time using autoencoders implemented on edge devices is exceedingly challenging due to limited hardware, energy, and computational resources. We show that these limitations can be addressed by designing an autoencoder with low-resolution non-volatile memory-based synapses and employing an effective quantized neural network learning algorithm. We further propose nanoscale ferromagnetic racetracks with engineered notches hosting magnetic domain walls (DW) as exemplary non-volatile memory-based autoencoder synapses, where limited state (5-state) synaptic weights are manipulated by spin orbit torque (SOT) current pulses to write different magnetoresistance states. The performance of anomaly detection of the proposed autoencoder model is evaluated on the NSL-KDD dataset. Limited resolution and DW device stochasticity aware training of the autoencoder is performed, which yields comparable anomaly detection performance to the autoencoder having floating-point precision weights. While the limited number of quantized states and the inherent stochastic nature of DW synaptic weights in nanoscale devices are typically known to negatively impact the performance, our hardware-aware training algorithm is shown to leverage these imperfect device characteristics to generate an improvement in anomaly detection accuracy (90.98%) compared to accuracy obtained with floating-point synaptic weights that are extremely memory intensive. Furthermore, our DW-based approach demonstrates a remarkable reduction of at least three orders of magnitude in weight updates during training compared to the floating-point approach, implying significant reduction in operation energy for our method. This work could stimulate the development of extremely energy efficient non-volatile multi-state synapse-based processors that can perform real-time training and inference on the edge with unsupervised data.

Hardware-algorithm co-design of Energy Efficient Federated Learning in Quantized Neural Network

The real-world implementation of federated learning (FL) for smart device-based applications necessitates energy-efficient convergent models due to resource-constrained devices. In the literature, deep learning (DL)–based models are frequently trained on FL to obtain high accuracy and quick convergence employing a 32-bit floating point precision level. However, such circumstances are not feasible for devices with limited resources. DL models demand a significant amount of processing power and energy, which has a noticeable negative impact on the environment. Hence, it is necessary to reduce the level of precision in DL models to enhance energy efficiency. In this paper, we propose a low-precision level green federated learning (GFL) model to maintain the balance of communication cycles, energy efficiency, and accuracy with an admissible convergence rate of the deep learning algorithms. Our proposed GFL framework achieves 98.04% server accuracy and 97.69% federated accuracy with a faster convergence rate and fewer communication rounds than state-of-the-art methods on the UCI human activity recognition dataset.

Theoretical and Practical Bounds on the Initial Value of Clock Skew Compensation Algorithm Immune to Floating-Point Precision Loss for Resource-Constrained Wireless Sensor Nodes

ABSTRACT We revisit our prior work on clock skew compensation immune to floating-point precision loss and provide practical as well as theoretical bounds on the initial value of the skew-compensated clock based on a systematic analysis of the errors of floating-point operations; by practical bounds, we mean the actual values of the theoretical bounds calculated at resource-constrained computing platforms like WSN nodes equipped with 32-bit single-precision floating-point format. Numerical examples demonstrate that the proposed practical bounds on single-precision floating-point format do not violate the theoretical bounds and thereby can guarantee the correctness of the clock skew compensation on resource-constrained computing platforms.

Hybrid Precision Floating-Point (HPFP) Selection to Optimize Hardware-Constrained Accelerator for CNN Training.

The rapid advancement in AI requires efficient accelerators for training on edge devices, which often face challenges related to the high hardware costs of floating-point arithmetic operations. To tackle these problems, efficient floating-point formats inspired by block floating-point (BFP), such as Microsoft Floating Point (MSFP) and FlexBlock (FB), are emerging. However, they have limited dynamic range and precision for the smaller magnitude values within a block due to the shared exponent. This limits the BFP's ability to train deep neural networks (DNNs) with diverse datasets. This paper introduces the hybrid precision (HPFP) selection algorithms, designed to systematically reduce precision and implement hybrid precision strategies, thereby balancing layer-wise arithmetic operations and data path precision to address the shortcomings of traditional floating-point formats. Reducing the data bit width with HPFP allows more read/write operations from memory per cycle, thereby decreasing off-chip data access and the size of on-chip memories. Unlike traditional reduced precision formats that use BFP for calculating partial sums and accumulating those partial sums in 32-bit Floating Point (FP32), HPFP leads to significant hardware savings by performing all multiply and accumulate operations in reduced floating-point format. For evaluation, two training accelerators for the YOLOv2-Tiny model were developed, employing distinct mixed precision strategies, and their performance was benchmarked against an accelerator utilizing a conventional brain floating point of 16 bits (Bfloat16). The HPFP selection, employing 10 bits for the data path of all layers and for the arithmetic of layers requiring low precision, along with 12 bits for layers requiring higher precision, results in a 49.4% reduction in energy consumption and a 37.5% decrease in memory access. This is achieved with only a marginal mean Average Precision (mAP) degradation of 0.8% when compared to an accelerator based on Bfloat16. This comparison demonstrates that the proposed accelerator based on HPFP can be an efficient approach to designing compact and low-power accelerators without sacrificing accuracy.

Numerical stability of DeepGOPlus inference.

Convolutional neural networks (CNNs) are currently among the most widely-used deep neural network (DNN) architectures available and achieve state-of-the-art performance for many problems. Originally applied to computer vision tasks, CNNs work well with any data with a spatial relationship, besides images, and have been applied to different fields. However, recent works have highlighted numerical stability challenges in DNNs, which also relates to their known sensitivity to noise injection. These challenges can jeopardise their performance and reliability. This paper investigates DeepGOPlus, a CNN that predicts protein function. DeepGOPlus has achieved state-of-the-art performance and can successfully take advantage and annotate the abounding protein sequences emerging in proteomics. We determine the numerical stability of the model's inference stage by quantifying the numerical uncertainty resulting from perturbations of the underlying floating-point data. In addition, we explore the opportunity to use reduced-precision floating point formats for DeepGOPlus inference, to reduce memory consumption and latency. This is achieved by instrumenting DeepGOPlus' execution using Monte Carlo Arithmetic, a technique that experimentally quantifies floating point operation errors and VPREC, a tool that emulates results with customizable floating point precision formats. Focus is placed on the inference stage as it is the primary deliverable of the DeepGOPlus model, widely applicable across different environments. All in all, our results show that although the DeepGOPlus CNN is very stable numerically, it can only be selectively implemented with lower-precision floating-point formats. We conclude that predictions obtained from the pre-trained DeepGOPlus model are very reliable numerically, and use existing floating-point formats efficiently.

Resource constrained neural network training

Modern applications of neural-network-based AI solutions tend to move from datacenter backends to low-power edge devices. Environmental, computational, and power constraints are inevitable consequences of such a shift. Limiting the bit count of neural network parameters proved to be a valid technique for speeding up and increasing efficiency of the inference process. Hence, it is understandable that a similar approach is gaining momentum in the field of neural network training. In the face of growing complexity of neural network architectures, reducing resources required for preparation of new models would not only improve cost efficiency but also enable a variety of new AI applications on modern personal devices. In this work, we present a deep refinement of neural network parameters limitation with the use of the asymmetric exponent method. In addition to the previous research, we study new techniques of floating-point variables limitation, representation, and rounding. Moreover, by leveraging exponent offset, we present floating-point precision adjustments without an increase in variables’ bit count. The proposed method allowed us to train LeNet, AlexNet and ResNet-18 convolutional neural networks with a custom 8-bit floating-point representation achieving minimal or no results degradation in comparison to baseline 32-bit floating-point variables.

Improved Design for Hardware Implementation of Graph-Based Large Margin Classifiers for Embedded Edge Computing.

The number of connected embedded edge computing Internet of Things (IoT) devices has been increasing over the years, contributing to the significant growth of available data in different scenarios. Thereby, machine learning algorithms arise to enable task automation and process optimization based on those data. However, due to some learning methods' computational complexity implementing geometric classifiers, it is a challenge to map these on embedded systems or devices with limited resources in size, processing, memory, and power, to accomplish the desired requirements. This hampers the applicability of these methods to complex industrial embedded edge applications. This work evaluates strategies to reduce classifiers' implementation costs based on the CHIP-clas model, independent of hyperparameter tuning and optimization algorithms. The proposal aims to evaluate the tradeoff between numerical precision and model performance and analyze the hardware implementations of a distance-based classifier. Two 16 -b floating-point formats were compared to the 32 -b floating-point precision implementation. Also, a new hardware architecture was developed and then compared to the state-of-the-art reference. The results indicate that the model is robust to low precision computation, providing statistically equivalent results compared to the baseline model, also pointing out statistically equivalent performance and a global speed-up factor of approx 4.39 in processing time.

Low-Power High Precision Floating-Point Divider With Bidimensional Linear Approximation

Low-Power High Precision Floating-Point Divider With Bidimensional Linear Approximation

On floating point precision in computational fluid dynamics using OpenFOAM

Thanks to the computational power of modern cluster machines, numerical simulations can provide, with an unprecedented level of details, new insights into fluid mechanics. However, taking full advantage of this hardware remains challenging since data communication remains a significant bottleneck to reaching peak performances. Reducing floating point precision is a simple and effective way to reduce data movement and improve the computational speed of most applications. Nevertheless, special care needs to be taken to ensure the quality and convergence of computed solutions, especially when dealing with complex fluid simulations. In this work, we analyse the impact of reduced (single and mixed compared to double) precision on computational performance and accuracy for computational fluid dynamics. Using the open source library OpenFOAM, we consider incompressible, compressible, and multiphase fluid solvers for testing on relevant benchmarks for flows in the laminar and turbulent regime and in the presence of shock waves. Computational gain and changes in the scalability of applications in reduced precision are also discussed. In particular, an ad hoc theoretical model for the strong scaling allows us to interpret and understand the observed behaviours, as a function of floating point precision and hardware specifics. Finally, we show how reduced precision can significantly speed up a hybrid CPU–GPU implementation, made available to OpenFOAM end-users recently, that simply relies on a GPU linear algebra solver developed by hardware vendors.

Mixed precision support in HPC applications: What about reliability?

In their quest for exascale and beyond, High-Performance Computing (HPC) systems continue becoming ever larger and more complex. Application developers, on the other hand, leverage novel methods to improve the efficiency of their own codes: a recent trend is the use of floating-point mixed precision, or the careful interlocking of single- and double-precision arithmetic, as a tool to improve performance as well as reduce network and memory boundedness. However, while it is known that modern HPC systems suffer hardware faults at daily rates, the impact of reduced precision on application reliability is yet to be explored. In this work we aim to fill this gap: first, we propose a qualitative survey to identify the branches of HPC where mixed precision is most popular. Second, we show the results of instruction-level fault injection experiments on a variety of representative HPC workloads, comparing vulnerability to Silent Data Errors (SDEs) under different numerical configurations. Our experiments indicate that use of single and mixed precision leads to comparatively more frequent and more severe SDEs, with concerning implications regarding their use on extreme-scale, fault-prone HPC platforms.