Fused Multiply-add Research Articles

Power Function Algorithms Implemented in Microcontrollers and FPGAs

The exponential function ax is widespread in many fields of science. Its calculation is a complicated issue for Central Processing Units (CPUs) and Graphics Processing Units (GPUs), as well as for specialised Digital Signal Processing (DSP) processors, such as Intelligent Processor Units (IPUs), for the needs of neural networks. This article presents some simple and accurate exponential function calculation algorithms in half, single, and double precision that can be prototyped in Field-Programmable Gate Arrays (FPGAs). It should be noted that, for the approximation, the use of effective polynomials of the first degree was proposed in most cases. The characteristic feature of such algorithms is that they only contain fast ‘bithack’ operations (‘bit manipulation technique’) and Floating-Point (FP) addition, multiplication, and (if necessary) Fused Multiply-Add (FMA) operations. We published an article on algorithms for this class of function recently, but the focus was on the use of approximations of second-degree polynomials and higher, requiring two multiplications and two additions or more, which poses some complications in FPGA implementation. This article considers algorithms based on piecewise linear approximation, with one multiplication and one addition. Such algorithms of low complexity provide decent accuracy and speed, sufficient for practical applications such as accelerators for neural networks, power electronics, machine learning, computer vision, and intelligent robotic systems. These are FP-oriented algorithms; therefore, we briefly describe the characteristic parameters of such numbers.

All-Digital Computing-in-Memory Macro Supporting FP64-Based Fused Multiply-Add Operation

Recently, frequent data movement between computing units and memory during floating-point arithmetic has become a major problem for scientific computing. Computing-in-memory (CIM) is a novel computing paradigm that merges computing logic into memory, which can address the data movement problem with excellent power efficiency. However, the previous CIM paradigm failed to support double-precision floating-point format (FP64) due to its computing complexity. This paper presents a novel all-digital CIM macro-DCIM-FF to complete FP64 based fused multiply-add (FMA) operation for the first time. With 16 sub-CIM cells integrating digital multipliers to complete mantissa multiplication, DCIM-FF is able to provide correct rounded implementations for normalized/denormalized inputs in round-to-nearest-even mode and round-to-zero mode, respectively. To evaluate our design, we synthesized and tested the DCIM-FF macro in 55-nm CMOS technology. With a minimum power efficiency of 0.12 mW and a maximum computing efficiency of 26.9 TOPS/W, we successfully demonstrated that DCIM-FF can run the FP64-based FMA operation without error. Compared to related works, the proposed DCIM-FF macro shows significant power efficiency improvement and less area overhead based on CIM technology. This work paves a novel pathway for high-performance implementation of an FP64-based matrix-vector multiplication (MVM) operation, which is essential for hyperscale scientific computing.

Matrix Multiplication in Multiword Arithmetic: Error Analysis and Application to GPU Tensor Cores

In multiword arithmetic, a matrix is represented as the unevaluated sum of two or more lower precision matrices, and a matrix product is formed by multiplying the constituents in low precision. We investigate the use of multiword arithmetic for improving the performance-accuracy tradeoff of matrix multiplication with mixed precision block fused multiply–add (FMA) hardware, focusing especially on the tensor cores available on NVIDIA GPUs. Building on a general block FMA framework, we develop a comprehensive error analysis of multiword matrix multiplication. After confirming the theoretical error bounds experimentally by simulating low precision in software, we use the cuBLAS and CUTLASS libraries to implement a number of matrix multiplication algorithms using double-fp16 (double-binary16) arithmetic. When running the algorithms on NVIDIA V100 and A100 GPUs, we find that double-fp16 is not as accurate as fp32 (binary32) arithmetic despite satisfying the same worst-case error bound. Using probabilistic error analysis, we explain why this issue is likely to be caused by the rounding mode used by the NVIDIA tensor cores, and we propose a parameterized blocked summation algorithm that alleviates the problem and significantly improves the performance-accuracy tradeoff.

A BF16 FMA is All You Need for DNN Training

Fused Multiply-Add (FMA) functional units constitute a fundamental hardware component to train Deep Neural Networks (DNNs). Its silicon area grows quadratically with the mantissa bit count of the computer number format, which has motivated the adoption of the BrainFloat16 format (BF16). BF16 features 1 sign, 8 exponent and 7 explicit mantissa bits. Some approaches to train DNNs achieve significant performance benefits by using the BF16 format. However, these approaches must combine BF16 with the standard IEEE 754 Floating-Point 32-bit (FP32) format to achieve state-of-the-art training accuracy, which limits the impact of adopting BF16. This article proposes the first approach able to train complex DNNs entirely using the BF16 format. We propose a new class of FMA operators, <inline-formula><tex-math notation="LaTeX">$\mathrm{FMA}^{\mathrm {bf}16}_{\mathrm{n}\_\mathrm{m}}$</tex-math></inline-formula> , that entirely rely on BF16 FMA hardware instructions and deliver the same accuracy as FP32. <inline-formula><tex-math notation="LaTeX">$\mathrm{FMA}^{\mathrm {bf}16}_{\mathrm{n}\_\mathrm{m}}$</tex-math></inline-formula> operators achieve performance improvements within the 1.28-1.35× range on ResNet101 with respect to FP32. <inline-formula><tex-math notation="LaTeX">$\mathrm{FMA}^{\mathrm {bf}16}_{\mathrm{n}\_\mathrm{m}}$</tex-math></inline-formula> enables training complex DNNs on simple low-end hardware devices without requiring expensive FP32 FMA functional units.

A Neural Network Training Processor With 8-Bit Shared Exponent Bias Floating Point and Multiple-Way Fused Multiply-Add Trees

Recent advances in deep neural networks (DNNs) and machine learning algorithms have induced the demand for services based on machine learning algorithms that require a large number of computations, and specialized hardware ranging from accelerators for data centers to on-device computing systems have been introduced. Low-precision math such as 8-bit integers have been used in neural networks for energy-efficient neural network inference, but training with low-precision numbers without performance degradation have remained to be a challenge. To overcome this challenge, this article presents an 8-bit floating-point neural network training processor for state-of-the-art non-sparse neural networks. As naïve 8-bit floating-point numbers are insufficient for training DNNs robustly, two additional methods are introduced to ensure high-performance DNN training. First, a novel numeric system which we dub as 8-bit floating point with shared exponent bias (FP8-SEB) is introduced. Moreover, multiple-way fused multiply-add (FMA) trees are used in FP8-SEB’s hardware implementation to ensure higher numerical precision and reduced energy. FP8-SEB format combined with multiple-way FMA trees is evaluated under various scenarios to show a trained-from-scratch performance that is close to or even surpasses that of current networks trained with full-precision (FP32). Our silicon-verified DNN training processor utilizes 24-way FMA trees implemented with FP8-SEB math and flexible 2-D routing schemes to show <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.48\times $ </tex-math></inline-formula> higher energy efficiency than prior low-power neural network training processors and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$78.1\times $ </tex-math></inline-formula> lower energy than standard GPUs.

Customizing FFT with FMA for China Accelerator

Recently, Fast Fourier Transform (FFT) was introduced to study pattern characterization in textiles. Accelerators are a promising platform to accelerate large-scale FFT computation. However, current accelerating FFT only uses the accelerator to compute, but uses a CPU as a mere task controller. Additionally, a specified interface for FFT is built into the China accelerator (CA), which severely restricts FFT parallelization and performance. Hence, we transform four multiplications and six additions into six Fused Multiply Add (FMA) operations, then reduce total float operations by 40%, assisted by FMAs equipped with a Vector Processing Unit (VPE). Moreover, we propose an Interface Adapter (IA) to cater to a specified interface and a Fused Algorithm for Interface Adapter (FAIA) to fully use both the CA and CPU to compute large-scale FFT with coordination. Experimental results validated successful performance.

Mixed Precision Block Fused Multiply-Add: Error Analysis and Application to GPU Tensor Cores

Computing units that carry out a fused multiply-add (FMA) operation with matrix arguments, referred to as tensor units by some vendors, have great potential for use in scientific computing. However, these units are inherently mixed precision, and existing rounding error analyses do not support them. We consider a mixed precision block FMA that generalizes both the usual scalar FMA and existing tensor units. We describe how to exploit such a block FMA in the numerical linear algebra kernels of matrix multiplication and LU factorization and give detailed rounding error analyses of both kernels. An important application is to GMRES-based iterative refinement with block FMAs, about which our analysis provides new insight. Our framework is applicable to the tensor core units in the NVIDIA Volta and Turing GPUs. For these we compare matrix multiplication and LU factorization with TC16 and TC32 forms of FMA, which differ in the precision used for the output of the tensor cores. Our experiments on an NVDIA V100 GPU confirm the predictions of the analysis that the TC32 variant is much more accurate than the TC16 one, and they show that the accuracy boost is obtained with almost no performance loss.

SIMD programming using Intel vector extensions

Single instruction multiple data (SIMD) extensions are one of the most significant capabilities of recent General Purpose Processors (GPPs) which improves the performance of applications with less hardware modification. Each GPP vendor such as HP, Sun, Intel, and AMD has its particular Instruction Set Architecture (ISA) and SIMD micro-architecture with different perspectives. Intel expanded SIMD technologies from hardware and software point of view. It has introduced SIMD technologies such as MultiMedia eXtensions (MMX), Streaming SIMD Extensions (SSE), Advanced Vector eXtensions (AVX), Fused Multiply Add (FMA) and AVX-512 sets. During micro-processors developments path, register width has been extended from 64 bits to 512 bits and number of vector registers has been increased from 8 to 32. Wider registers provide more parallelism ways and more registers reduce extra data movement to the cache memory. In order to gain the advantages of SIMD extensions, many programming approaches have been developed. Compiler Automatic Vectorization (CAV) as an implicit vectorization approach, provides simple and easy SIMDization tools. While, performance improvement of CAV is not always granted, most compilers auto-vectorize simple loops. On the other hand, for explicit vectorization, Intrinsic Programming Model (IPM) provides low-level access to vector registers for SIMDizing. However, programming with IPM requires great amount of expertise especially in low-level architecture feature, thus, choosing the suitable instructions and vectorization methodology for mapping to a certain algorithm is important. Moreover, portability, compatibility, scalability and compiler optimization might limit the advantage of IPM. Our goal in this paper is as follows. First, we provide a review of SIMD technology in general and Intel’s SIMD extensions in particular. Second, some SIMD features of Intel SIMD technologies, MMX, SSEs, AVX, and FMA in terms of ISA, vector width, and SIMD programming tools are comparatively discussed. Third, in order to compare the performance of different auto-vectorizers and IPM approaches using Intel C++ compiler (ICC), GNU Compiler Collection (GCC) and Low Level Virtual Machine (LLVM), we map and implement some representative multimedia kernels on AVX and AVX2 extensions. Finally, our experimental results show that although the performance improvement using IPM approach is higher than CAVs, programmer needs more programming efforts and knows different mapping strategists. Therefore, extending auto-vectorizers abilities to generate more efficient vectorized codes is an important issue in different compilers.

Identifying volatile numeric expressions in numeric computing applications

The results of numerical computations with floating point numbers depend on the execution platform, which we define as the hardware and the tools (compilers, etc.) supporting that hardware. One reason for the dependence is that compilers have significant freedom in deciding how to evaluate a floating point expression, as such evaluation is not standardized (not even in standards such as IEEE-754). Another reason is that hardware may or may not provide specialized instructions like Fused Multiply Add (FMA), and if it does, the compiler can take advantage of FMA functionality in different ways.We call an expression volatile if, for some input, its value differs across platform parameters. Differences can become particularly large across heterogeneous parallel architectures. This undermines the software portability promised by programming standards such as OpenCL and significantly impacts reproducibility of results in general.In this paper, we present a technique that predicts bounds on the output of a program containing volatile expressions when executed on different platforms. Using randomly selected inputs, we compare the bounds to results from running the code across a variety of platforms including CPUs and GPUs. Our results show that the theoretical bounds are relatively tight (within an order of magnitude) and can help users pinpoint where results should be stabilized, for instance by restricting expression reordering.

Vector Processing-Aware Advanced Clock-Gating Techniques for Low-Power Fused Multiply-Add

The need for power efficiency is driving a rethink of design decisions in processor architectures. While vector processors succeeded in the high-performance market in the past, they need a retailoring for the mobile market that they are entering now. Floating-point (FP) fused multiply-add (FMA), being a functional unit with high power consumption, deserves special attention. Although clock gating is a well-known method to reduce switching power in synchronous designs, there are unexplored opportunities for its application to vector processors, especially when considering active operating mode. In this research, we comprehensively identify, propose, and evaluate the most suitable clock-gating techniques for vector FMA units (VFUs). These techniques ensure power savings without jeopardizing the timing. We evaluate the proposed techniques using both synthetic and “real-world” application-based benchmarking. Using vector masking and vector multilane-aware clock gating, we report power reductions of up to 52%, assuming active VFU operating at the peak performance. Among other findings, we observe that vector instruction-based clock-gating techniques achieve power savings for all vector FP instructions. Finally, when evaluating all techniques together, using “real-world” benchmarking, the power reductions are up to 80%. Additionally, in accordance with processor design trends, we perform this research in a fully parameterizable and automated fashion.

Efficient dual-precision floating-point fused-multiply-add architecture

The fused-multiply-add (FMA) instruction is a common instruction in RISC processors since 1990. A 3-stage, 8-level pipelined, dual-precision FMA is proposed here that can perform operations either at one double precision (SISD) or at two single precision in parallel (SIMD). The 53-bit mantissa-multiplier (MM) is optimally segmented by Karatsuba–Offman (KO) algorithm such that both modes can be performed. The 6-stage pipelined MM uses only 6 of 10 multipliers and 13 of 33 adder/subtractors in SIMD. Thus hardware area of the proposed MM is reduced by 23.82% and throughput is maintained to be 923M samples/s. The arithmetic operational units in the data path are shared among the modes by having four data rearrangement units (DRU) which rearranges the data systematically at the input, the outputs of MM and the final output. Though these DRUs bring some hardware overhead, the resulting architecture is modular and uniform for both modes of computation. The proposed FMA has been implemented using TSMC 1P6M CMOS 130 nm library and takes 48% less overall area and consumes 49% less power at 308.7 MHz compared to previous results. The area-delay-product (ADP), 0.48 × 10−15 shows that the area optimization by proposed KO based MM can also keep the computation time as 3.24 ns.

Fully GPU‐based electromagnetic transient simulation considering large‐scale control systems for system‐level studies

As more generators and loads are integrated by power electronic converters with complicated controls, electromagnetic transients (EMTs) simulation becomes an important tool for studying dynamic characteristics of large-scale power systems. To accelerate system-level EMT simulations, a fine-grained parallel algorithm on graphics processing units (GPU) is proposed. By decomposing the computational models of the EMT simulation into heterogeneous, homogeneous and network solution computations, the simulations are mapped into three unified GPU kernels. To incorporate control signals and non-linear features of electrical components, heterogeneous computations are formulated as layered direct acyclic graphs (LDAG) of primitive operations. An LDAG kernel is designed to carry out theses primitive operations efficiently by grouped threads. Then, homogeneous computations for state updates of electrical components are modelled as sets of fused multiply-add (FMA) operations, which are concurrently processed by an FMA kernel. Moreover, a hybrid network solution kernel is designed to solve the network equations, which can adaptively select dense or sparse solvers. Large-scale test systems are created and simulated on an NVIDIA K20x GPU. The results show that the proposed GPU-based EMT simulations are accurate and achieve 10x speedups over the CPU-based ones.

Fast Combined Decimal/Binary Multiplier Based on Redundant BCD 4221-8421Digit Recoding

Many applications consider floating point arithmeticas a key component of the computations. Combineddecimal/binary arithmetic becomes an important topicsupports high-speed decimal/binary applications. A new 64-bit(16×16 digit) the combined decimal/binary multiplier is proposedand implemented in this work that can be used for both fusedmultiply-add (FMA) and multiplier unit. A new partialproducts reduction tree is shared between decimal and binarymultiplier unit. The valuation and comparison result betweenthe proposed multiplier and the previous most recent worksshows 4.66 % less delay than combined decimal/binarymultiplier and 19.33 % less delay than the fastest standalonedecimal multiplier.

Hybrid Hardware/Software Floating-Point Implementations for Optimized Area and Throughput Tradeoffs

Hybrid floating-point (FP) implementations improve software FP performance without incurring the area overhead of full hardware FP units. The proposed implementations are synthesized in 65-nm CMOS and integrated into small fixed-point processors with a RISC-like architecture. Unsigned, shift carry, and leading zero detection (USL) support is added to a processor to augment an existing instruction set architecture and increase FP throughput with little area overhead. The hybrid implementations with USL support increase software FP throughput per core by $2.18\times $ for addition/subtraction, $1.29\times $ for multiplication, 3.07– $4.05\times $ for division, and 3.11– $3.81\times $ for square root, and use 90.7–94.6% less area than dedicated fused multiply-add (FMA) hardware. Hybrid implementations with custom FP-specific hardware increase throughput per core over a fixed-point software kernel by 3.69– $7.28\times $ for addition/subtraction, 1.22– $2.03\times $ for multiplication, $14.4\times $ for division, and $31.9\times $ for square root, and use 77.3–97.0% less area than dedicated FMA hardware. The circuit area and throughput are found for 38 multiply-add, 8 addition/subtraction, 6 multiplication, 45 division, and 45 square root designs. Thirty-three multiply-add implementations are presented, which improve throughput per core versus a fixed-point software implementation by 1.11– $15.9\times $ and use 38.2–95.3% less area than dedicated FMA hardware.

Area Efficient and Fast Combined Binary/Decimal Floating Point Fused Multiply Add Unit

In this work we present a new 64-bit floating point Fused Multiply Add (FMA) unit that can perform both binary and decimal addition, multiplication, and fused-multiply-add operations. The presented FMA has 6 percent less delay than the fastest stand-alone decimal unit and 23 percent less area than both binary and decimal units together. These results were achieved by the use of: 1) column by column reduction to reduce the partial products in the multiplier tree, 2) a new leading zeros detector that produces its output in base-3 to simplify the normalization shifting in the binary datapath, 3) the use of a redundant adder to perform the final addition, 4) using a new rounding-while-redundant technique to hide the rounding delay and remove it from the critical path, and 5) using a new simple conversion technique from redundant to binary/decimal.

Estimating numerical error in neural network simulations on Graphics Processing Units

Modern graphics processing units (GPUs) are becoming a popular hardware substrate for spiking neural network simulations [1-3], due to their massive parallelism and impressive cost-to-speed ratio. However, verifying and interpreting the results of a GPU simulation can be difficult because the results are never exactly reproducible, unlike those from an equivalent serial simulation on a CPU; not only is the simulation subject to the usual rounding errors of floating-point arithmetic, but there are also elements of stochasticity due to the non-determinism of the thread scheduling mechanism on the hardware, alongside the non-associativity of floating-point addition and multiplication. Consider, for example, a typical postsynaptic integration step for the summation of incoming synapse currents, executed on a GPU. If there are multiple threads, which are each simulating an incoming synapse, there is no guarantee for the order in which each synapse thread's current will be accumulated into the total current. Therefore, rounding errors will be different depending on this order and the result could be different every time the simulation runs. Such effects would initially be small but can be amplified in unstable or chaotic systems to a degree that the final results appear completely random across different runs (see Figure ​Figure11). Figure 1 In repeated runs, results of numerical simulations on GPUs can vary. Mean, standard deviation and range of observed membrane potential of a neuron in a network of 10,000 Izhikevich neurons, 8,000 excitatory and 2,000 inhibitory, with 1,000 random connections ... When comparing runs between GPU and CPU implementations there are additional sources of divergence. There are subtle differences in the way each architecture implements floating-point arithmetic. For instance, the NVIDIA C2070 GPU tested in this study implements the fused multiply-add (FMA) operation, introduced in the latest IEEE-754-2008 floating-point standard, whereas most Intel CPUs perform the multiplication and addition operations separately, with lower accuracy. Only Intel's most recent Haswell CPU architecture implements the more accurate FMA operation, whilst many current lab workstations contain chips that do not. The aim of the work presented here is to analytically determine the theoretical worst-case and average-case numerical absolute error incurred when simulating neural network models on an NVIDIA CUDA GPU. These error measurements are also compared with the absolute error resulting from the equivalent serial algorithm running on a single CPU core, using standard float32 (float) and float64 (double) precision floating-point arithmetic, to determine a reasonable error margin for verifying the results of parallel GPU simulations against those of equivalent serial CPU simulations. Furthermore, both CPU and GPU implementations are compared against an equivalent simulation using an accurate arbitrary-precision floating-point arithmetic library, to determine how far the CPU and GPU simulation trajectories deviate from the analytically 'correct' trajectory. For illustration, the divergence of a single neuron in a 10,000 neuron Izhikevich network is plotted in the figure. Finally, we also analyse the role of errors originating from approximate integration methods and compare them to the underlying numerical errors discussed thus far.

Speculative hardware/software co-designed floating-point multiply-add fusion

A Fused Multiply-Add (FMA) instruction is currently available in many general-purpose processors. It increases performance by reducing latency of dependent operations and increases precision by computing the result as an indivisible operation with no intermediate rounding. However, since the arithmetic behavior of a single-rounding FMA operation is different than independent FP multiply followed by FP add instructions, some algorithms require significant revalidation and rewriting efforts to work as expected when they are compiled to operate with FMA--a cost that developers may not be willing to pay. Because of that, abundant legacy applications are not able to utilize FMA instructions. In this paper we propose a novel HW/SW collaborative technique that is able to efficiently execute workloads with increased utilization of FMA, by adding the option to get the same numerical result as separate FP multiply and FP add pairs. In particular, we extended the host ISA of a HW/SW co-designed processor with a new Combined Multiply-Add (CMA) instruction that performs an FMA operation with an intermediate rounding. This new instruction is used by a transparent dynamic translation software layer that uses a speculative instruction-fusion optimization to transform FP multiply and FP add sequences into CMA instructions. The FMA unit has been slightly modified to support both single-rounding and double-rounding fused instructions without increasing their latency and to provide a conservative fall-back path in case of mispeculation. Evaluation on a cycle-accurate timing simulator showed that CMA improved SPECfp performance by 6.3% and reduced executed instructions by 4.7%.

Speculative hardware/software co-designed floating-point multiply-add fusion

A Fused Multiply-Add (FMA) instruction is currently available in many general-purpose processors. It increases performance by reducing latency of dependent operations and increases precision by computing the result as an indivisible operation with no intermediate rounding. However, since the arithmetic behavior of a single-rounding FMA operation is different than independent FP multiply followed by FP add instructions, some algorithms require significant revalidation and rewriting efforts to work as expected when they are compiled to operate with FMA--a cost that developers may not be willing to pay. Because of that, abundant legacy applications are not able to utilize FMA instructions. In this paper we propose a novel HW/SW collaborative technique that is able to efficiently execute workloads with increased utilization of FMA, by adding the option to get the same numerical result as separate FP multiply and FP add pairs. In particular, we extended the host ISA of a HW/SW co-designed processor with a new Combined Multiply-Add (CMA) instruction that performs an FMA operation with an intermediate rounding. This new instruction is used by a transparent dynamic translation software layer that uses a speculative instruction-fusion optimization to transform FP multiply and FP add sequences into CMA instructions. The FMA unit has been slightly modified to support both single-rounding and double-rounding fused instructions without increasing their latency and to provide a conservative fall-back path in case of mispeculation. Evaluation on a cycle-accurate timing simulator showed that CMA improved SPECfp performance by 6.3% and reduced executed instructions by 4.7%.

Mechanical Derivation of Fused Multiply–Add Algorithms for Linear Transforms

Several computer architectures offer fused multiply-add (FMA), also called multiply-and-accumulate (MAC) instructions, that are as fast as a single addition or multiplication. For the efficient implementation of linear transforms, such as the discrete Fourier transform or discrete cosine transforms, this poses a challenge to algorithm developers as standard transform algorithms have to be manipulated into FMA algorithms that make optimal use of FMA instructions. We present a general method to convert any transform algorithm into an FMA algorithm. The method works with both algorithms given as directed acyclic graphs (DAGs) and algorithms given as structured matrix factorizations. We prove bounds on the efficiency of the method. In particular, we show that it removes all single multiplications except at most as many as the transform has outputs. We implemented the DAG-based version of the method and show that we can generate many of the best-known hand-derived FMA algorithms from the literature as well as a few novel FMA algorithms.

MULTIPLY-ADD OPTIMIZED FFT KERNELS

Modern computer architecture provides a special instruction — the fused multiply-add (FMA) instruction — to perform both a multiplication and an addition operation at the same time. In this paper newly developed radix-2, radix-3, and radix-5 FFT kernels that efficiently take advantage of this powerful instruction are presented. If a processor is provided with FMA instructions, the radix-2 FFT algorithm introduced has the lowest complexity of all Cooley–Tukey radix-2 algorithms. All floating-point operations are executed as FMA instructions. Compared to conventional radix-3 and radix-5 kernels, the new radix-3 and radix-5 kernels greatly improve the utilization of FMA instructions, which results in a significant reduction in complexity. In general, the advantages of the FFT algorithms presented in this paper are their low arithmetic complexity, their high efficiency, and their striking simplicity. Numerical experiments show that FFT programs using the new kernels clearly outperform even the best conventional FFT routines.