Field Programmable Gate Array Design Research Articles

Quantization-Aware NN Layers with High-throughput FPGA Implementation for Edge AI

Over the past few years, several applications have been extensively exploiting the advantages of deep learning, in particular when using convolutional neural networks (CNNs). The intrinsic flexibility of such models makes them widely adopted in a variety of practical applications, from medical to industrial. In this latter scenario, however, using consumer Personal Computer (PC) hardware is not always suitable for the potential harsh conditions of the working environment and the strict timing that industrial applications typically have. Therefore, the design of custom FPGA (Field Programmable Gate Array) solutions for network inference is gaining massive attention from researchers and companies as well. In this paper, we propose a family of network architectures composed of three kinds of custom layers working with integer arithmetic with a customizable precision (down to just two bits). Such layers are designed to be effectively trained on classical GPUs (Graphics Processing Units) and then synthesized to FPGA hardware for real-time inference. The idea is to provide a trainable quantization layer, called Requantizer, acting both as a non-linear activation for neurons and a value rescaler to match the desired bit precision. This way, the training is not only quantization-aware, but also capable of estimating the optimal scaling coefficients to accommodate both the non-linear nature of the activations and the constraints imposed by the limited precision. In the experimental section, we test the performance of this kind of model while working both on classical PC hardware and a case-study implementation of a signal peak detection device running on a real FPGA. We employ TensorFlow Lite for training and comparison, and use Xilinx FPGAs and Vivado for synthesis and implementation. The results show an accuracy of the quantized networks close to the floating point version, without the need for representative data for calibration as in other approaches, and performance that is better than dedicated peak detection algorithms. The FPGA implementation is able to run in real time at a rate of four gigapixels per second with moderate hardware resources, while achieving a sustained efficiency of 0.5 TOPS/W (tera operations per second per watt), in line with custom integrated hardware accelerators.

A New Recursive Trigonometric Technique for FPGA-Design Implementation

This paper presents a new recursive trigonometric (RT) technique for Field-Programmable Gate Array (FPGA) design implementation. The traditional implementation of trigonometric functions on FPGAs requires a significant amount of data storage space to store numerous reference values in the lookup tables. Although the coordinate rotation digital computer (CORDIC) can reduce the required FPGA storage space, their implementation process can be very complex and time-consuming. The proposed RT technique aims to provide a new approach for generating trigonometric functions to improve communication accuracy and reduce response time in the FPGA. This new RT technique is based on the trigonometric transformation; the output is calculated directly from the input values, so its accuracy depends only on the accuracy of the inputs. The RT technique can prevent complex iterative calculations and reduce the computational errors caused by the scale factor K in the CORDIC. Its effectiveness in generating highly accurate cosine waveform is verified by simulation tests undertaken on an FPGA.

A Domain-Specific Accelerator for Ultralow Latency Market Data Distribution System

Ultralow latency parsing of financial data is gaining significance in the high-frequency trading of the security exchange market. Hardware-aided systems exhibit superior improvement of latency over traditional software solutions, but flexibility may suffer when processing the financial protocol. This article presents a domain-specific accelerator for the market data distribution system, which integrates a financial information exchange adapted for streaming (FAST) decoder, a 10-Gbps network interface, and a high-speed PCIe host interface into a single field programmable gate array (FPGA) acceleration card. The proposed FAST decoder adopts the finite state machines-coordinated sequence mapping table to achieve run-time reconfigurability over fine-grained FPGA programming, and 16 fields can be decoded simultaneously in a pipelined manner, resulting in a decoding latency of only 33 ns. Evaluated on the Xilinx Alveo U200 acceleration card, this work improves the latency of decoding a FAST message by 26%–72% than state-of-the-art FPGA designs, and outperforms the software baseline by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$&gt;27\times$</tex-math></inline-formula> in terms of latency covering both decoding and communication.

FCRoute: A Fast FPGA Connection Router Using Soft Routing-Space Pruning Algorithm

Routing is one of the most time-consuming stages in the field-programmable gate array (FPGA) design flow. Even if various attempts have been made to reduce route time, the existing work rarely focuses on improving the underlying A*-based FPGA connection router. In this article, we present a fast FPGA connection router called FCRoute based on a novel soft routing-space pruning algorithm. Within FCRoute, a routing resource priority mechanism is applied to classify the routing resource nodes into high-priority nodes and low-priority ones. On the whole, FCRoute is composed of fast maze search and backtracking process. During the fast maze search, we explore only the high-priority nodes in the routing space. In this way, a great deal of unnecessary work can be avoided. When the fast maze search fails to find the target sink, it allows the backtracking process to explore the low-priority nodes promising to be on the best path, after which a new fast maze search is called. By avoiding the exploration of the majority of low-priority nodes, FCRoute maintains runtime efficiency while ensuring global search ability. In addition, we further accelerate FCRoute with an engineering enhancement which simplifies the cost computations of nodes. Runtime and quality of results are compared with the state-of-the-art connection router in VPR 8. Experimental results show that on average FCRoute explores less than half the number of routing resource nodes, and therefore reduces runtime by 38% while enabling the quality of results. When combined with the enhancement, FCRoute achieves an average 45% reduction on runtime without sacrificing the quality of results.

Hardware Design and Implementation of FPGA Controlled Seven-Level Reduced Switch MLI

The paper proposes the design of a seven-level reduced switch multi-level inverter (7LRS-MLI) topology. The design produces a seven-level staircase output waveform with fewer components (seven power switches), thus reducing the cost and area and improving the performance of the design, i.e. increasing efficiency and reducing switching loss effectively. Both symmetric and asymmetric configurations have been implemented and the results are discussed in the paper. A comparison of the proposed structure with other recent seven-level structures has been illustrated. The simulation (using MATLAB/SIMULINK software) and experimental validation of 7LRS-MLI also have been performed for R, RL load, change in load and change in a modulation index (MI). Field programmable gate arrays (FPGA) have been used for digital implementation of the switching waveforms which reduces design time and improves flexibility. For switching devices, the alternative phase opposite disposition pulse width modulation (APOD-PWM) technique is used at 500 Hz and 5 kHz. The voltage and current total harmonic distortion (THD) profiles are also computed during the simulation and experimental validation.

Design of novel fractional order FPGA based reactor protection and safety controllers for ACP1000 nuclear power plant in LabVIEW

In this research work, an advanced most modern ACP1000 Nuclear Power Plant is addressed. An enhanced fractional order model of ACP1000 nuclear power plant is adopted with an addition of protection and safety systems. The whole plant model is developed by using innovative hybrid technology of Visual Basic, LabVIEW, Fractional Order and Field Programmable Gate Array (FPGA). A reactor trip system is designed and modeled using FPGA technology in LabVIEW. Plant parameters are systematically modeled and panels are designed in LabVIEW for reactor protection controllers. Twenty one reactor trip controllers are designed and modeled based on complex digital logics using FPGA programming in LabVIEW. Two fractional order trip controllers are designed for over temperature protection and over power protection in LabVIEW. FPGA based safety controllers are designed for Engineered Safety Features (ESF) in LabVIEW. For enhanced model of ACP1000 nuclear power plant, 374 systems are modeled in modular form in Visual Basic Environment. Nine process controllers are configured in ANFIS framework in LabVIEW. In this research work, process controllers are used in conjunction with protection and safety controllers using FPGA. The parametric display of simulations is carried out in Visual Basic. The closed loop performance of proposed protection controllers is evaluated under reactor trip and turbine trip while that of safety controllers are evaluated under inadvertent opening of safety valves of pressurizer. Various parameters are simulated for severe transient conditions and the results are evaluated and validated against reference design data and Final Safety Analysis Report (FSAR) of ACP1000 nuclear power plant. All the results are well within the trip and safety systems design bounds under abnormal and severe operating conditions.

An integrated and scalable experimental system for nitrogen-vacancy ensemble magnetometry.

Nitrogen-vacancy (NV) centers in diamond are extremely promising solid-state spin quantum sensors for magnetic field in recent years. The rapid development of NV-ensemble magnetometry has put forward higher requirements for high-speed data acquisition, real-time signal processing and analyzing, etc. However, the existing commercial instruments are bulky and expensive, which brings extra complexity to the weak magnetic field detection experiment and hinders the practicality and miniaturization of NV-ensemble magnetometry. Here, we report on an integrated and scalable experimental system based on a field-programmable-gate-array (FPGA) chip assisted with high-speed peripherals for NV-ensemble magnetometry, which presents a compact and compatible design containing high-speed data acquisition, oscilloscopes, signal generator, spectrum analyzer, lock-in amplifier, proportional-integral-derivative feedback controller, etc. To verify its applicability and reliability in experiments, various applications, such as optical magnetic resonance detection, optical cavity locking, and lock-in NV magnetometry, are conducted. We further realize the pump-enhanced magnetometry based on NV center ensembles using the optical cavity. Through the flexible FPGA design approach, this self-developed device can also be conveniently extended into atomic magnetometer and other quantum systems.

A Sparse CNN Accelerator for Eliminating Redundant Computations in Intra- and Inter-Convolutional/Pooling Layers

Neural network pruning, which can be divided into unstructured pruning and structured pruning strategies, has been proven to be an efficient method to substantially reduce the number of computations of convolutional neural networks (CNNs). However, it remains difficult to combine the advantages of these two pruning strategies. This article proposes a high-performance accelerator for unstructured sparse CNNs. First, a convolution-based filter selection and clustering method (FSCM) is proposed to reorder unstructured sparse filters into uniform-size dense filters, eliminating redundant computations in the convolutional layers while maintaining a regular structure. In addition, a convolution and pooling calculation method (CPCM) is presented to reduce interlayers’ computational redundancy. Third, a hardware accelerator with high digital signal processing (DSP) efficiency is designed to take advantage of FSCM and CPCM. The proposed accelerator is implemented on a Xilinx XCVU9P platform at 300 MHz. With different CNN configurations, the performance and DSP efficiency for LeNet, AlexNet, and VGG16 are 1490.53 GOPS and 3.882, 913.85 GOPS and 1.785, and 862.16 GOPS and 1.68, respectively. Compared to previous field-programmable gate array (FPGA) and application-specific integrated circuit (ASIC) designs, the accelerator achieves up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$15.43\times $ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$7.73\times $ </tex-math></inline-formula> speedup in terms of performance and DSP efficiency.

Importance of Edge Computing in Critical Manufacturing Systems: FPGA Implementation

Academic and industrial studies on smart systems, which have entered all areas of our lives, continue to their rapid developments along with gaining momentum with Industry 4.0. Especially, many of the different production devices with their molding, printing, shaping, and cutting capabilities have a certain level of automation. They work together with the material information they process, and the compatibility with other machines to make the whole production system work effectively and properly. While the abundance of data acquired is an important source for better analytics, obtaining information for business purposes from this data and helping decision support systems is the most important task expected from Information Technology and Systems in the organization. In this paper, we propose an FPGA-based edge information infrastructure to evaluate critical data from the production devices, distributed sensors, and other ISs in any industrial environment to increase the utilization and performance of the total machinery. This study helps the predictive maintenance decision for a sample plastic injection molding device according to our industrial scenario. A sample data set downloaded from the Internet with the factors like speed, vibration, and the temperature was used. An FPGA (Field Programmable Gate Array) design that will run the necessary ML algorithms with the sensor data and existing information system inputs (ERP, MES) has been carried out by using Xilinx Design Tools and Vitis IDE 2020.2. In this study, the ANFIS (Adaptive Network-Based Fuzzy Inference System) system, which is an approach consisting of the integration of artificial neural networks and Fuzzy Logic, has been chosen as an Artificial Intelligence application. The estimation results obtained were evaluated over the accuracy rates achieved in similar studies in the literature.

New Real-Time Impulse Noise Removal Method Applied to Chest X-ray Images.

In this paper, we propose a new Modified Laplacian Vector Median Filter (MLVMF) for real-time denoising complex images corrupted by "salt and pepper" impulsive noise. The method consists of two rounds with three steps each: the first round starts with the identification of pixels that may be contaminated by noise using a Modified Laplacian Filter. Then, corrupted pixels pass a neighborhood-based validation test. Finally, the Vector Median Filter is used to replace noisy pixels. The MLVMF uses a 5 × 5 window to observe the intensity variations around each pixel of the image with a rotation step of π/8 while the classic Laplacian filters often use rotation steps of π/2 or π/4. We see better identification of noise-corrupted pixels thanks to this rotation step refinement. Despite this advantage, a high percentage of the impulsive noise may cause two or more corrupted pixels (with the same intensity) to collide, preventing the identification of noise-corrupted pixels. A second round is then necessary using a second set of filters, still based on the Laplacian operator, but allowing focusing only on the collision phenomenon. To validate our method, MLVMF is firstly tested on standard images, with a noise percentage varying from 3% to 30%. Obtained performances in terms of processing time, as well as image restoration quality through the PSNR (Peak Signal to Noise Ratio) and the NCD (Normalized Color Difference) metrics, are compared to the performances of VMF (Vector Median Filter), VMRHF (Vector Median-Rational Hybrid Filter), and MSMF (Modified Switching Median Filter). A second test is performed on several noisy chest x-ray images used in cardiovascular disease diagnosis as well as COVID-19 diagnosis. The proposed method shows a very good quality of restoration on this type of image, particularly when the percentage of noise is high. The MLVMF provides a high PSNR value of 5.5% and a low NCD value of 18.2%. Finally, an optimized Field-Programmable Gate Array (FPGA) design is proposed to implement the proposed method for real-time processing. The proposed hardware implementation allows an execution time equal to 9 ms per 256 × 256 color image.

Constrained Optimization of FPGA Design for Spaceborne InSAR Processing

With the development of spaceborne processing technologies, the demand for on-board processing has risen sharply. Against this background, spaceborne Interferometric Synthetic Aperture Radar (InSAR) processing has become an important research area. In many cases, high processing capacity is required during on-board InSAR processing, yet Field-Programmable Gate Array (FPGA) resources on the satellites are limited. To improve the performance of spaceborne remote sensing processing, this paper designs a high-performing FPGA system for the coarse registration and interferogram generation process of InSAR. Moreover, to address this dual-constraint problem of resource and processing capacity, the paper proposes an FPGA design method based on the gradient descent theory, which can identify the optimum trade-off scheme between two such constraints. Finally, the proposed system design and method are implemented in FPGA. Experiments showed that the FPGA system outperformed the NVIDIA (Santa Clara, CA, USA) GTX Titan Black Graphics Processing Unit (GPU), and the optimum trade-off scheme only increases the entire time by 1.1% but reduces the FPGA BRAM usage by 8.7%. The experimental results proved the effectiveness and validity of the proposed system and method.

Development of a Signal Processing Software for Scintillation Detectors and Implementation on an FPGA for Fast Sensing

An open-source code RadSigPro 1.0 has been developed and used for fast processing of nanosecond-long pulses from scintillation detectors. This processing includes pulse height distribution (PHD), pulse shape discrimination (PSD), and time of flight (TOF). The code has been implemented onto the programmable logic design of a field programmable gate array (FPGA) design for on-the-fly processing of neutron and gamma-ray pulses. A weighted average of the percent difference of the results for RadSigPro 1.0 implemented on a CPU and a FPGA logic design is calculated. This shows a 0% difference for the PHD data sets, a 0.458% and 0.344% difference for the designated gamma detector and neutron detector PSD data sets, respectively, and a 0% difference for the TOF data set. When the FPGA logic design is applied and simulated, it computed the total and tail pulse areas within 5 ns of the arrival of the final data point used for accumulation and also captured the pulse height value within 2 ns of the arrival of the pulse’s maximum data point.

A Comprehensive Methodology to Optimize FPGA Designs via the Roofline Model

With reconfigurable fabrics delivering increasing performance over the years, Field-Programmable Gate Arrays (FPGAs) are becoming an appealing solution for next-generation High-Performance Computing (HPC) systems. However, in order to gain traction among traditional von Neumann architectures, the optimization process of Field-Programmable Gate Array (FPGA) designs should be further abstracted to a higher level. In fact, while High-Level Synthesis (HLS) already provides a handy way to write FPGA code with common high-level languages, substantial effort and expertise are still required to optimize the resulting FPGA design for the underlying hardware. To overcome this problem, we propose a semi-automated performance optimization methodology based on a Hierarchical Roofline model for FPGAs. System-wide and applications-specific optimizations such as off-chip memory transfer and data locality optimizations are guided by the FPGA Roofline model whereas FPGA-specific optimizations are automatically searched by a Design Space Exploration (DSE) engine. We demonstrate the way this methodology allows to easily analyze and optimize to peak system performance a wide set of applications ranging from particle methods, wavefront algorithms, and sparse arithmetic computations. In addition, we prove that the integrated Design Space Exploration (DSE) engine achieves a 14.36x maximum speedup if compared to previous automated solutions in the literature.

HXDP

The network interface cards (NICs) of modern computers are changing to adapt to faster data rates and to help with the scaling issues of general-purpose CPU technologies. Among the ongoing innovations, the inclusion of programmable accelerators on the NIC's data path is particularly interesting, since it provides the opportunity to offload some of the CPU's network packet processing tasks to the accelerator. Given the strict latency constraints of packet processing tasks, accelerators are often implemented leveraging platforms such as Field-Programmable Gate Arrays (FPGAs). FPGAs can be re-programmed after deployment, to adapt to changing application requirements, and can achieve both high throughput and low latency when implementing packet processing tasks. However, they have limited resources that may need to be shared among diverse applications, and programming them is difficult and requires hardware design expertise. We present hXDP, a solution to run on FPGAs software packet processing tasks described with the eBPF technology and targeting the Linux's eXpress Data Path. hXDP uses only a fraction of the available FPGA resources, while matching the performance of high-end CPUs. The iterative execution model of eBPF is not a good fit for FPGA accelerators. Nonetheless, we show that many of the instructions of an eBPF program can be compressed, parallelized, or completely removed, when targeting a purpose-built FPGA design, thereby significantly improving performance. We implement hXDP on an FPGA NIC and evaluate it running real-world unmodified eBPF programs. Our implementation runs at 156.25MHz and uses about 15% of the FPGA resources. Despite these modest requirements, it can run dynamically loaded programs, achieves the packet processing throughput of a high-end CPU core, and provides a 10X lower packet forwarding latency.

Field-programmable gate array design of image encryption and decryption using Chua’s chaotic masking

&lt;span&gt;This article presents a simple and efficient masking technique based on Chua chaotic system synchronization. It includes feeding the masked signal back to the master system and using it to drive the slave system for synchronization purposes. The proposed system is implemented in a field programmable gate array (FPGA) device using the Xilinx system generator tool. To achieve synchronization, the Pecora-Carroll identical cascading synchronization approach was used. The transmitted signal should be mixed or masked with a chaotic carrier and can be processed by the receiver without any distortion or loss. For different images, the security analysis is performed using the histogram, correlation coefficient, and entropy. In addition, FPGA hardware co-simulation based Xilinx Artix7 xc7a100t-1csg324 was used to check the reality of the encryption and decryption of the images.&lt;/span&gt;

STT-MRAM-Based Multicontext FPGA for Multithreading Computing Environment

The demand for high-performance computing and rapidly increasing power consumption has increased the necessity for application-specific accelerators. In the datacenter and mobile system, more applications are increasingly relying on accelerators. Field-programmable gate arrays (FPGAs) emerge as a good candidate because they have high programmability and power efficiency. As the number of applications requiring acceleration increases, there is huge demand for FPGAs that support multiple contexts. Previous FPGA designs that support multicontext have various shortcomings such as volatility, poor power efficiency, large performance, area, and reconfiguration overhead. In this article, we propose a spin-transfer torque magnetic RAM (STT-MRAM)-based nonvolatile multicontext FPGA (NVMC-FPGA) that overcomes these shortcomings. We introduce the NVMC-FPGA architecture and operation modes that take advantage of nonvolatility and support multicontext. We also develop the multicontext-aware FPGA computer aided design flow to make the most of the NVMC-FPGA. Compared to the conventional SRAM-based FPGA, when eight identical circuits are mapped, the NVMC-FPGA improves the performance by 15.3% on average and reduces the power consumption by 11.2%–80.7%, depending on the number of simultaneously activated circuits. Moreover, when eight different circuits are mapped, the NVMC-FPGA improves the performance by 58.5% on average and reduces the power consumption by 6.2%–63.3%, depending on the number of simultaneously activated circuits.

FPGA Implementation for the Sigmoid with Piecewise Linear Fitting Method Based on Curvature Analysis

The sigmoid activation function is popular in neural networks, but its complexity limits the hardware implementation and speed. In this paper, we use curvature values to divide the sigmoid function into different segments and employ the least squares method to solve the expressions of the piecewise linear fitting function in each segment. We then adopt an optimization method with maximum absolute errors and average absolute errors to select an appropriate function expression with a specified number of segments. Finally, we implement the sigmoid function on the field-programmable gate array (FPGA) development platform and apply parallel operations of arithmetic (multiplying and adding) and range selection at the same time. The FPGA implementation results show that the clock frequency of our design is up to 208.3 MHz, while the end-to-end latency is just 9.6 ns. Our piecewise linear fitting method based on curvature analysis (PWLC) achieves recognition accuracy on the MNIST dataset of 97.51% with a deep neural network (DNN) and 98.65% with a convolutional neural network (CNN). Experimental results demonstrate that our FPGA design of sigmoid function can obtain the lowest latency, reduce absolute errors, and achieve high recognition accuracies, while the hardware cost is acceptable in practical applications.

Resources and Power Efficient FPGA Accelerators for Real-Time Image Classification.

A plethora of image and video-related applications involve complex processes that impose the need for hardware accelerators to achieve real-time performance. Among these, notable applications include the Machine Learning (ML) tasks using Convolutional Neural Networks (CNNs) that detect objects in image frames. Aiming at contributing to the CNN accelerator solutions, the current paper focuses on the design of Field-Programmable Gate Arrays (FPGAs) for CNNs of limited feature space to improve performance, power consumption and resource utilization. The proposed design approach targets the designs that can utilize the logic and memory resources of a single FPGA device and benefit mainly the edge, mobile and on-board satellite (OBC) computing; especially their image-processing- related applications. This work exploits the proposed approach to develop an FPGA accelerator for vessel detection on a Xilinx Virtex 7 XC7VX485T FPGA device (Advanced Micro Devices, Inc, Santa Clara, CA, USA). The resulting architecture operates on RGB images of size or sliding windows; it is trained for the “Ships in Satellite Imagery” and by achieving frequency 270 MHz, completing the inference in 0.687 ms and consuming 5 watts, it validates the approach.

FPGA Accelerator for Real-Time Non-Line-of-Sight Imaging

Non-line-of-sight (NLOS) imaging systems reconstruct hidden scenes using computational methods based on indirect light that diffusely reflected from relay walls. Due to the computation and memory requirements of reconstruction algorithms, real-time NLOS imaging for room-size scenes based on non-confocal data has long been challenging. This paper proposes a field programmable gate array (FPGA) accelerator for the recently proposed Rayleigh-Sommerfeld Diffraction (RSD)-based NLOS reconstruction method. In the proposed accelerator design, ring sampling and radius sampling techniques are proposed to reduce the memory requirements by reconstructing the RSD kernels with a set of kernel bases and ring sampling coefficients during the runtime. Based on that, a customized hardware architecture and the corresponding FPGA design for real-time RSD-based NLOS reconstruction is further proposed. Implementation results show that the proposed FPGA accelerator is capable of reconstructing NLOS scenes at 25 frames per second (FPS), running at a relatively slow clock frequency of 50 MHz. To the best knowledge of the authors, this is the first real-time enabled FPGA accelerator for room-size NLOS imaging with a resolution of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$128\times 128$ </tex-math></inline-formula> .

XDNN: Inference for Deep Convolutional Neural Networks

We present xDNN, an end-to-end system for deep-learning inference based on a family of specialized hardware processors synthesized on Field-Programmable Gate Array (FPGAs) and Convolution Neural Networks (CNN). We present a design optimized for low latency, high throughput, and high compute efficiency with no batching. The design is scalable and a parametric function of the number of multiply-accumulate units, on-chip memory hierarchy, and numerical precision. The design can produce a scale-down processor for embedded devices, replicated to produce more cores for larger devices, or resized to optimize efficiency. On Xilinx Virtex Ultrascale+ VU13P FPGA, we achieve 800 MHz that is close to the Digital Signal Processing maximum frequency and above 80% efficiency of on-chip compute resources. On top of our processor family, we present a runtime system enabling the execution of different networks for different input sizes (i.e., from 224× 224 to 2048× 1024). We present a compiler that reads CNNs from native frameworks (i.e., MXNet, Caffe, Keras, and Tensorflow), optimizes them, generates codes, and provides performance estimates. The compiler combines quantization information from the native environment and optimizations to feed the runtime with code as efficient as any hardware expert could write. We present tools partitioning a CNN into subgraphs for the division of work to CPU cores and FPGAs. Notice that the software will not change when or if the FPGA design becomes an ASIC, making our work vertical and not just a proof-of-concept FPGA project. We show experimental results for accuracy, latency, and power for several networks: In summary, we can achieve up to 4 times higher throughput, 3 times better power efficiency than the GPUs, and up to 20 times higher throughput than the latest CPUs. To our knowledge, we provide solutions faster than any previous FPGA-based solutions and comparable to any other top-of-the-shelves solutions.