Intel CPU Research Articles

A FPGA Accelerator of Distributed A3C Algorithm with Optimal Resource Deployment

The asynchronous advantage actor-critic (A3C) algorithm is widely regarded as one of the most effective and powerful algorithms among various deep reinforcement learning algorithms. However, the distributed and asynchronous nature of the A3C algorithm brings increased algorithm complexity and computational requirements, which not only leads to an increased training cost but also amplifies the difficulty of deploying the algorithm on resource-limited field programmable gate array (FPGA) platforms. In addition, the resource wastage problem caused by the distributed training characteristics of A3C algorithms and the resource allocation problem affected by the imbalance between the computational amount of inference and training need to be carefully considered when designing accelerators. In this paper, we introduce a deployment strategy designed for distributed algorithms aimed at enhancing the resource utilization of hardware devices. Subsequently, a FPGA architecture is constructed specifically for accelerating the inference and training processes of the A3C algorithm. The experimental results show that our proposed deployment strategy reduces resource consumption by 62.5% and decreases the number of agents waiting for training by 32.2%, and the proposed A3C accelerator achieves 1.83× and 2.39× improvements in speedup compared to CPU (Intel i9-13900K) and GPU (NVIDIA RTX 4090) with less power consumption respectively. Furthermore, our design shows superior resource efficiency compared to existing works.

UniSparse: An Intermediate Language for General Sparse Format Customization

The ongoing trend of hardware specialization has led to a growing use of custom data formats when processing sparse workloads, which are typically memory-bound. These formats facilitate optimized software/hardware implementations by utilizing sparsity pattern- or target-aware data structures and layouts to enhance memory access latency and bandwidth utilization. However, existing sparse tensor programming models and compilers offer little or no support for productively customizing the sparse formats. Additionally, because these frameworks represent formats using a limited set of per-dimension attributes, they lack the flexibility to accommodate numerous new variations of custom sparse data structures and layouts. To overcome this deficiency, we propose UniSparse, an intermediate language that provides a unified abstraction for representing and customizing sparse formats. Unlike the existing attribute-based frameworks, UniSparse decouples the logical representation of the sparse tensor (i.e., the data structure) from its low-level memory layout, enabling the customization of both. As a result, a rich set of format customizations can be succinctly expressed in a small set of well-defined query, mutation, and layout primitives. We also develop a compiler leveraging the MLIR infrastructure, which supports adaptive customization of formats, and automatic code generation of format conversion and compute operations for heterogeneous architectures. We demonstrate the efficacy of our approach through experiments running commonly-used sparse linear algebra operations with specialized formats on multiple different hardware targets, including an Intel CPU, an NVIDIA GPU, an AMD Xilinx FPGA, and a simulated processing-in-memory (PIM) device.

Crystal Polymorph Search in the NPT Ensemble via a Deposition/Sublimation Alchemical Path.

The formulation of active pharmaceutical ingredients involves discovering stable crystal packing arrangements or polymorphs, each of which has distinct pharmaceutically relevant properties. Traditional experimental screening techniques utilizing various conditions are commonly supplemented with in silico crystal structure prediction (CSP) to inform the crystallization process and mitigate risk. Predictions are often based on advanced classical force fields or quantum mechanical calculations that model the crystal potential energy landscape but do not fully incorporate temperature, pressure, or solution conditions during the search procedure. This study proposes an innovative alchemical path that utilizes an advanced polarizable atomic multipole force field to predict crystal structures based on direct sampling of the NPT ensemble. The use of alchemical (i.e., nonphysical) intermediates, a novel Monte Carlo barostat, and an orthogonal space tempering bias combine to enhance the sampling efficiency of the deposition/sublimation phase transition. The proposed algorithm was applied to 2-((4-(2-(3,4-dichlorophenyl)ethyl)phenyl)amino)benzoic acid (Cambridge Crystallography Database Centre ID: XAFPAY) as a case study to showcase the algorithm. Each experimentally determined polymorph with one molecule in the asymmetric unit was successfully reproduced via approximately 1000 short 1 ns simulations per space group where each simulation was initiated from random rigid body coordinates and unit cell parameters. Utilizing two threads of a recent Intel CPU (a Xeon Gold 6330 CPU at 2.00 GHz), 1 ns of sampling using the polarizable AMOEBA force field can be acquired in 4 h (equating to more than 300 ns/day using all 112 threads/56 cores of a dual CPU node) within the Force Field X software (https://ffx.biochem.uiowa.edu). These results demonstrate a step forward in the rigorous use of the NPT ensemble during the CSP search process and open the door to future algorithms that incorporate solution conditions using continuum solvation methods.

BestCRM: An Exhaustive Search for Optimal Cis-Regulatory Modules in Promoters Accelerated by the Multidimensional Hash Function.

The concept of cis-regulatory modules located in gene promoters represents today's vision of the organization of gene transcriptional regulation. Such modules are a combination of two or more single, short DNA motifs. The bioinformatic identification of such modules belongs to so-called NP-hard problems with extreme computational complexity, and therefore, simplifications, assumptions, and heuristics are usually deployed to tackle the problem. In practice, this requires, first, many parameters to be set before the search, and second, it leads to the identification of locally optimal results. Here, a novel method is presented, aimed at identifying the cis-regulatory elements in gene promoters based on an exhaustive search of all the feasible modules' configurations. All required parameters are automatically estimated using positive and negative datasets. To be computationally efficient, the search is accelerated using a multidimensional hash function, allowing the search to complete in a few hours on a regular laptop (for example, a CPU Intel i7, 3.2 GH, 32 Gb RAM). Tests on an established benchmark and real data show better performance of BestCRM compared to the available methods according to several metrics like specificity, sensitivity, AUC, etc. A great practical advantage of the method is its minimum number of input parameters-apart from positive and negative promoters, only a desired level of module presence in promoters is required.

3D-ShuffleViT: An Efficient Video Action Recognition Network with Deep Integration of Self-Attention and Convolution

Compared with traditional methods, the action recognition model based on 3D convolutional deep neural network captures spatio-temporal features more accurately, resulting in higher accuracy. However, the large number of parameters and computational requirements of 3D models make it difficult to deploy on mobile devices with limited computing power. In order to achieve an efficient video action recognition model, we have analyzed and compared classic lightweight network principles and proposed the 3D-ShuffleViT network. By deeply integrating the self-attention mechanism with convolution, we have introduced an efficient ACISA module that further enhances the performance of our proposed model. This has resulted in exceptional performance in both context-sensitive and context-independent action recognition, while reducing deployment costs. It is worth noting that our 3D-ShuffleViT network, with a computational cost of only 6% of that of SlowFast-ResNet101, achieved 98% of the latter’s Top1 accuracy on the EgoGesture dataset. Furthermore, on the same CPU (Intel i5-8300H), its speed was 2.5 times that of the latter. In addition, when we deployed our model on edge devices, our proposed network achieved the best balance between accuracy and speed among lightweight networks of the same order.

Cross Layer Design Using HW/SW Co-Design and HLS to Accelerate Chaining in Genomic Analysis

DNA sequence analysis is a computationally intensive task. Minimap2 is a state-of-the-art software tool for third-generation sequence analysis workflow. Nearly 50% of Minimap2’s computation time is spent on what is known as the chaining step. In this paper, the chaining step is accelerated using a novel heterogeneous computing system combining an Intel FPGA-based hardware accelerator and a CPU (using HLS for the FPGA and multi-threaded software for the CPU). The system in this paper is capable of handling large realistic workloads, achieves up to ~1.35× performance improvement over the software solution running on an Intel CPU with SIMD intrinsics (Intel’s latest AVX-512), while consuming ~27% less energy. When compared to the software solution running on the CPU without SIMD intrinsics, the system performs ~1.9× faster while consuming ~38% less energy. Importantly, this work also ensures that the accuracy of the output generated is not compromised for the speed-up gained (this work only has an error rate of less than 10-6% compared to 26% in previous FPGA-based chaining step accelerator).

An Intermediate Language for General Sparse Format Customization

The inevitable trend of hardware specialization drives an increasing use of custom data formats in processing sparse workloads, which are typically memory-bound. These formats facilitate the automated generation of target-aware data layouts to improve memory access latency and bandwidth utilization. However, existing sparse tensor programming models and compilers offer little or no support for productively customizing the sparse formats. Moreover, since these frameworks adopt an attribute-based approach for format abstraction, they cannot easily be extended to support general format customization. To overcome this deficiency, we propose UniSparse, an intermediate language that provides a unified abstraction for representing and customizing sparse formats. More concretely, we express a sparse format as a map from dense coordinates to a layout tree using a small set of well-defined query and mutation primitives. We also develop a compiler leveraging the MLIR infrastructure, which supports adaptive customization of formats, and automatic code generation of format conversion and compute operations for heterogeneous architectures. We demonstrate the efficacy of our approach through experiments running commonly-used sparse linear algebra operations with hybrid formats on multiple different hardware targets, including an Intel CPU, an NVIDIA GPU, and a simulated processing-in-memory (PIM) device.

BOUNCE: memory-efficient SIMD approach for lightweight integer compression

Integer compression plays an important role in columnar database systems to reduce the main memory footprint as well as to speedup query processing. To keep the additional computational effort of (de)compression as low as possible, the powerful Single Instruction Multiple Data (SIMD) extensions of modern CPUs are heavily applied. While a scalar compression algorithm usually compresses a block of N consecutive integers, the state-of-the-art SIMDified implementation scales the block size to k cdot N with k as the number of elements which could be simultaneously processed in an SIMD register. On the one hand, this scaling SIMD approach improves the performance of (de)compression. But on the other hand, it can lead to a degradation of the memory footprint of the compressed data. Within this article, we analyze this degradation effect for various integer compression algorithms and present a novel SIMD concept to overcome that effect. The core idea of our novel SIMD concept called BOUNCE is to concurrently compress k different blocks of size N within SIMD registers, guaranteeing the same compression ratio as scalar variant. As we are going to show, our proposed SIMD idea works well on various Intel CPUs and may offer a new generalized SIMD concept to optimize further algorithms.

The Opportunities and Limitations of Extended Page Table Switching for Fine-Grained Isolation

Extended Page Table switching with VMFUNC is a hardware isolation mechanism available in Intel CPUs. VMFUNC is attractive for low overhead and the possibility to isolate privileged kernel code. However, many careful design decisions are needed to ensure the security of the isolation boundary.

Automatic Creation of High-bandwidth Memory Architectures from Domain-specific Languages: The Case of Computational Fluid Dynamics

Numerical simulations can help solve complex problems. Most of these algorithms are massively parallel and thus good candidates for FPGA acceleration thanks to spatial parallelism. Modern FPGA devices can leverage high-bandwidth memory technologies, but when applications are memory-bound designers must craft advanced communication and memory architectures for efficient data movement and on-chip storage. This development process requires hardware design skills that are uncommon in domain-specific experts. In this article, we propose an automated tool flow from a domain-specific language for tensor expressions to generate massively parallel accelerators on high-bandwidth-memory-equipped FPGAs. Designers can use this flow to integrate and evaluate various compiler or hardware optimizations. We use computational fluid dynamics (CFD) as a paradigmatic example. Our flow starts from the high-level specification of tensor operations and combines a multi-level intermediate representation–based compiler with an in-house hardware generation flow to generate systems with parallel accelerators and a specialized memory architecture that moves data efficiently, aiming at fully exploiting the available CPU-FPGA bandwidth. We simulated applications with millions of elements, achieving up to 103 GFLOPS with one compute unit and custom precision when targeting a Xilinx Alveo U280. Our FPGA implementation is up to 25× more energy efficient than expert-crafted Intel CPU implementations.

Optimizing the performance of a server-based classification for a large business document flow

The document categorization problem in the case of a large business document flow is considered. Textual and visual embeddings were employed for classification. Textual embeddings were extracted via OCR Tesseract. The Viola and Jones method was applied to generate visual embeddings. This paper describes the performance optimization technology for the implemented classification algorithm. Servers with Intel CPUs were used for the algorithm execution. For single-threaded implementation, high-level and low-level optimizations were performed. High-level optimization was based on the parametrization of the recognition algorithms and the employment of intermediate data. Low-level optimization was carried out via compiler tools allowing for an extended set of SIMD instructions. The implementation of parallelization with several multithreaded applications on multiple servers was also described. The proposed solution was tested using own test data sets of business documents. The proposed method can be applied in modern information systems to analyze the content of a large flow of digital document images.

НЕДОКУМЕНТИРОВАННЫЕ ИНСТРУКЦИИ Х86 ДЛЯ УПРАВЛЕНИЯ CPU НА УРОВНЕ МИКРОАРХИТЕКТУРЫ В СОВРЕМЕННЫХ ПРОЦЕССОРАХ INTEL

The purpose of this study was to uncover previously unknown vulnerabilities in Intel CPUs caused by implementation errors or backdoors embedded in system firmware, applications, and hardware. The authors have discovered the Red Unlocked debugging mode which allows microcode to be extracted from Intel Atom processors. Using this debugging mode, the internal microcode structure and the implementation of x86 instructions have been examined, and two undocumented x86 instructions were found. These undocumented x86 instructions, udbgrd and udbgwr, can read and write microarchitectural data. These instructions are assumed to be intended for Intel engineers to debug the CPU microarchitecture. However, their existence poses a cybersecurity threat: there is a working demonstration available in the public domain on how to activate the Red Unlock mode for one of the current Intel platforms. This paper presents the analysis of the udbgrd and udbgwr instructions and explains the conditions under which they can be used on commonly available platforms. This kind of research can be used to develop methods, tools, and solutions to ensure information security of systems and networks by countering threats that arise from newly identified vulnerabilities stemming from implementation defects or backdoors in system firmware, applications, and hardware.

RDPNet: a single-path lightweight CNN with re-parameterization for CPU-type edge devices

Deep convolutional neural networks have produced excellent results when utilized for image classification tasks, and they are being applied in a growing number of contexts. Model inference on edge devices is challenging due to the unending complicated structures needed to improve performance, which adds a significant computing burden.According to recent research, the often utilized residual structure in models does not support model inference. The idea of structural reparameterization is put out to address this shortcoming. The RepVGG produced with this method is a high-performance, quick-inference single-path network. Even after reparameterization, the model still needs GPUs and other specialized computing libraries to accelerate inference, however this still has a limit on how quickly the model can infer at the edge. We construct RDPNet using depthwise separable convolution and structural reparameterization to further reduce model size and accelerate inference. When utilizing an Intel CPU, this is a straightforward network that may be utilized for inference. For re-parameterization, we specifically adopt Depthwise separable convolution as the basic convolution form. Create a multi-branch model for training on the training side, and then simplify it into a single-branch model that the edge devices can easily infer. Research demonstrates that compared to alternative lightweight networks that can attain SOTA performance, RDPNet offers a superior trade-off between accuracy and latency.

An FPGA-based Convolution IP Core for Deep Neural Networks Acceleration

The development of machine learning has made a revolution in various applications such as object detection, image/video recognition, and semantic segmentation. Neural networks, a class of machine learning, play a crucial role in this process because of their remarkable improvement over traditional algorithms. However, neural networks are now going deeper and cost a significant amount of computation operations. Therefore they usually work ineffectively in edge devices that have limited resources and low performance. In this paper, we research a solution to accelerate the neural network inference phase using FPGA-based platforms. We analyze neural network models, their mathematical operations, and the inference phase in various platforms. We also profile the characteristics that affect the performance of neural network inference. Based on the analysis, we propose an architecture to accelerate the convolution operation used in most neural networks and takes up most of the computations in networks in terms of parallelism, data reuse, and memory management. We conduct different experiments to validate the FPGA-based convolution core architecture as well as to compare performance. Experimental results show that the core is platform-independent. The core outperforms a quad-core ARM processor functioning at 1.2 GHz and a 6-core Intel CPU with speed-ups of up to 15.69× and 2.78×, respectively

Validation of Deep Learning-Based DFCNN in Extremely Large-Scale Virtual Screening and Application in Trypsin I Protease Inhibitor Discovery.

Computational methods with affordable computational resources are highly desirable for identifying active drug leads from millions of compounds. This requires a model that is both highly efficient and relatively accurate, which cannot be achieved by most of the current methods. In real virtual screening (VS) application scenarios, the desired method should perform much better in selecting active compounds by prediction than by random chance. Here, we systematically evaluate the performance of our previously developed DFCNN model in large-scale virtual screening, and the results show our method has approximately 22 times the success rate compared to the random chance on average with a score cutoff of 0.99. Of the 102 test cases, 10 cases have more than 98 times the success rate of a random guess. Interestingly, in three cases, the prediction success rate is 99 times that of a random guess by a score cutoff of 0.99. This indicates that in most situations after our extremely large-scale VS, the dataset can be reduced 20 to 100 times for the next step of virtual screening based on docking or MD simulation. Furthermore, we have employed an experimental method to verify our computational method by finding several activity inhibitors for Trypsin I Protease. In addition, we also show its proof-of-concept application in de novo drug screening. The results indicate the massive potential of this method in the first step of the real drug development workflow. Moreover, DFCNN only takes about 0.0000225s for one protein–compound prediction on average with 80 Intel CPU cores (2.00 GHz) and 60 GB RAM, which is at least tens of thousands of times faster than AutoDock Vina or Schrödinger high-throughput virtual screening. Additionally, an online webserver based on DFCNN for large-scale screening is available at http://cbblab.siat.ac.cn/DFCNN/index.php for the convenience of the users.

ИССЛЕДОВАНИЕ ОГРАНИЧЕНИЙ ПРИМЕНИМОСТИ МИКРОПРОЦЕССОРОВ РЯДА ЭЛЬБРУС ДЛЯ РЕШЕНИЯ ЗАДАЧ ТЕХНИЧЕСКОГО ЗРЕНИЯ

One of the key areas in the artificial intelligence is technical vision. For resource-intensivetasks of technical vision high-performance, computing systems are created with use of specializedaccelerators. The use of such accelerators is necessary due to the inability of general-purposemicroprocessors (GPM) to solve such problems in a given time due to a high computational load.However, the microprocessors of Elbrus series are successfully used to solve technical visionproblems in both server and on-board modes, and the appearance of the sixth-generation Elbrusmicroprocessors should further improve performance on such tasks. Due to the high cost, greatercomplexity and limitations in the use of systems with specialized accelerators, the question arisesof determining the conditions under which, it is sufficient to use CPU’s to solve the tasks of technicalvision, for example, with the microprocessors of the Elbrus series without special accelerators.One of the most resource-intensive tasks in the field of technical vision are detection andclassification of objects. For the detection of objects one of the popular methods is the Viola-Jonesmethod. Convolutional neural networks are usually used to solve the classification problem.Mathematical models of computations have been developed for VGG16 and VGG19 neural networksin relation to the actual microprocessors of the Elbrus series. Using the developed models,the theoretical sufficiency of the performance of Elbrus microprocessors for technical vision tasksis substantiated. Also, based on these methods, programs for modeling detection and classificationsobjects in the image and video stream have been developed. The programs are written inC++ using the OpenCV library, OPO Elbrus, the GNS Platform library and the ImageNet competitiondatabase. Using the implemented programs, comparative testing was carried out on a numberof high-performance computing systems with Elbrus and Intel CPU’s and NVidia video card.Based on the results obtained, it is shown that the Elbrus-8S is sufficient to solve the problem ofsearching for objects in the image for input resolutions up to 1920 x 1080, where the processingspeed of the video stream is more than 20 frames per second.

A Configurable and Real-Time Multi-Frequency 3D Image Signal Processor for Indirect Time-of-Flight Sensors

3D time-of-flight (ToF) cameras have recently attracted widespread attention because of their potential in wide applications. However, the lack of versatile 3D image signal processor (ISP) is still an obstacle that hinders their popularity. The main challenges lie in the management of high-throughput raw data and the complexity of 3D image processing in hardware. To address these challenges, this paper presents a top-down 3D ISP with high real-time performance and high energy efficiency for indirect ToF (IToF) imaging. First, we propose configurable cache strategies to improve the efficiency of non-streaming memory accesses and support different IToF sensors. Then, to resolve the phase ambiguity problem commonly existing in IToF sensors, we develop a hardware-optimized cascaded multi-frequency fusion method, which not only supports up to six modulation frequencies but also facilitates the pixel-wise detection of multipath interference. Moreover, we propose a scalable calculation engine for depth calculation and calibration, 3D image enhancement, and post-processing. The pipeline is prototyped on a Xilinx ZCU102 FPGA board. Results show that the 3D ISP has negligible precision loss and low latency, achieving <inline-formula> <tex-math notation="LaTeX">$53.35\times $ </tex-math></inline-formula> energy efficiency improvement compared with the implementation on Intel CPU and <inline-formula> <tex-math notation="LaTeX">$55.45\times $ </tex-math></inline-formula> improvement over the ARM CPU implementation. When running at 75 MHz, the 3D ISP can support <inline-formula> <tex-math notation="LaTeX">$1280\times 960$ </tex-math></inline-formula> depth, intensity, and point cloud output at a frame rate of 30 f/s.

An innovative hybrid hydrometallurgical approach for precious metals recovery from secondary resources

The current paper presents an innovative hydrometallurgical methodology for recovery of precious metals from different waste streams. The used hydrometallurgical process, which has been already patented (Birloaga and Francesco Veglio, 2019), consists of a single leaching system (HCl, H2O2 and C2H4O2) of all elements and then selective recovery of elements from solution by chemical reduction/processes. About 99% of Au dissolution efficiency was achieved using: 3.5 M HCl; 1.96 M H2O2; 1.67 M of C2H4O2; 5 h without stirring; room temperature; 20% of solid concentration. The same conditions have resulted recovery of over 95% of Au from spent mobile phones PCBs. Over 80% of Au was achieved by three steps of leaching of ceramic Intel CPU. The influences of hydrochloric acid concentration and process time have been evaluated for Pd and Pt leaching from spent autocatalyst. Over 89% of Pt and 100% of Pd recovery were obtained from spent catalyst using: 5 M HCl; 1.96 M H2O2; 1.67 M of C2H4O2; 250 rpm of stirring for 3 h; room temperature; 10% of solid concentration. More than 98% of Pd recovery was achieved from the turbine residue after 20 min of reaction. The almost complete recovery of Au (99%) from solution was achieved by reduction with ascorbic acid. Complete recovery of Pd and about 79% of Pt have been obtained by cementation with ion metal powder from the leaching solution of spent autocatalyst. The application of reduction process with iron metal on the solution of turbine residues led to over 99% of Pd recovery. This efficient process provides a new way to recycle precious metals and to effectively prevent environmental pollution from different e-waste and other waste streams.

3D Convolutional Neural Network for Speech Emotion Recognition With Its Realization on Intel CPU and NVIDIA GPU

Due to the high level of precision and remarkable capabilities to solve the intricate problems in industry and academia, convolutional neural networks (CNNs) are presented. Speech emotion recognition is an interesting application for CNNs in the field of audio processing. In this paper, a speech emotion recognition system based on a 3D CNN is suggested to analyze and classify the emotions. In the proposed method, the three-dimensional reconstructed phase spaces of the speech signals were calculated. Then, emotion-related patterns formed in these spaces were converted into 3D tensors. Accordingly, a 3D CNN for speech emotion recognition applied to two datasets, EMO-DB and eNTERFACE05, using a speaker-independent technique achieved 90.40% and 82.20% accuracy, respectively. By employing gender recognition, the accuracy rates on EMO-DB increased to 94.42% and on eNTERFACE05 rose to 88.47%. Realization of the introduced 3D CNN on both Intel CPU and NVIDIA GPU is also explored. The results of the implemented 3D CNN without and with regard to gender recognition show that GPU-based running is faster for the EMO-DB and eNTERFACE05 datasets than CPU-based executions (using Python).

ThermalBleed: A Practical Thermal Side-Channel Attack

Modern OSs expose an interface for monitoring CPU temperature to unprivileged users for effective user decision-based thermal management. Due to the low sampling rate and resolution, thermal sensors have generally been restricted to the construction of covert channels. However, exposing the thermal interface to unprivileged users may be problematic, because the heat emission inside a CPU core is affected by program execution on the core; an attacker may be able to infer the secret information of the program by exploiting the thermal interface as a side-channel. In this paper, we extensively analyze digital thermal sensors in Intel CPUs and show that it is possible to implement a software-based thermal side-channel attack. Specifically, by analyzing some properties of the thermal sensors, we inferred that the thermal sensor makes it possible to distinguish between a cache hit and a physical memory access in memory load operations. Based on the analysis results, we implement ThermalBleed, a thermal side-channel attack that breaks kernel address space layout randomization (KASLR) in Linux systems. Moreover, by conducting an in-depth analysis, we identify useful hidden properties of the Intel thermal sensors. Our analysis establishes a stepping stone to build a more precise and effective thermal side-channel attack in the future. To the best of our knowledge, this is the first work that extends a thermal covert channel to a practical side-channel attack by exploring the properties of Intel digital thermal sensors.