Branch Prediction Research Articles

Semi-static conditions in low-latency C++ for high frequency trading: Better than branch prediction hints

Conditional branches pose a challenge for code optimisation, particularly in low latency settings. We present a novel language construct, referred to as a semi-static condition, which enables programmers to dynamically modify the direction of a branch at run-time by modifying the assembly code within the underlying executable. Subsequently, we explore scenarios where the use of semi-static conditions outperforms traditional conditional branching, highlighting their potential applications in real-time machine learning and high-frequency trading (HFT). Throughout the development process, key considerations of performance, portability, syntax, and security were taken into account.

Resource‐adaptive and OOD‐robust inference of deep neural networks on IoT devices

AbstractEfficiently executing inference tasks of deep neural networks on devices with limited resources poses a significant load in IoT systems. To alleviate the load, one innovative method is branching that adds extra layers with classification exits to a pre‐trained model, enabling inputs with high‐confidence predictions to exit early, thus reducing inference cost. However, branching networks, not originally tailored for IoT environments, are susceptible to noisy and out‐of‐distribution (OOD) data, and they demand additional training for optimal performance. The authors introduce BrevisNet, a novel branching methodology designed for creating on‐device branching models that are both resource‐adaptive and noise‐robust for IoT applications. The method leverages the refined uncertainty estimation capabilities of Dirichlet distributions for classification predictions, combined with the superior OOD detection of energy‐based models. The authors propose a unique training approach and thresholding technique that enhances the precision of branch predictions, offering robustness against noise and OOD inputs. The findings demonstrate that BrevisNet surpasses existing branching techniques in training efficiency, accuracy, overall performance, and robustness.

WCET Analysis Based on Micro-Architecture Modeling for Embedded System Security

To ensure the timely execution of hard real-time applications, scheduling analysis techniques must consider safe upper bounds on the possible execution durations of tasks or runnables, which are referred to as Worst-Case Execution Times (WCET). Bounding WCET requires not only program path analysis but also modeling the impact of micro-architectural features present in modern processors. In this paper, we model the ARMv8 ISA and micro-architecture including instruction cache, branch predictor, instruction prefetching strategies, out-of-order pipeline. We also consider the complex interactions between these features (e.g., cache misses caused by branch predictions and branch misses caused by instruction pipelines) and estimate the WCET of the program using the Implicit Path Enumeration Technique (IPET) static WCET analysis method. We compare the estimated WCET of benchmarks with the observed WCET on two ARMv8 boards. The ratio of estimated to observed WCET values for all benchmarks is greater than 1, demonstrating the security of the analysis.

Design of RISC-V out-of-order processor based on segmented exclusive or Gshare branch prediction

To address the balanced requirements of performance, power consumption, and area in embedded systems, this paper introduces a 32-bit out-of-order processor based on the RISC-V instruction set, and the processor supports interrupt handling. This processor is designed to support the RV32IMC instruction subset and utilizes a four-stage pipeline structure featuring sequential instruction fetching, out-of-order execution, and out-of-order write-back. The main contributions are as follows:1) The segmented exclusive or G-share branch prediction scheme for embedded processors is proposed, which provides high branch prediction accuracy when the capacity of pattern history table (PHT) is small. 2) The work undertaken by hardware and software during interrupt response is reasonably divided, which allows for fast interrupt response with minimal resource consumption. Furthermore, the interrupt response time in vectored mode is reduced by simultaneously stacking the control and status registers (CSRs) and obtaining the interrupt service routine (ISR) entry address. Executing branch prediction-related programs within an identical environment, the processor detailed in this paper demonstrates an average prediction accuracy improvement of 1.2 % over G-share and 0.6 % over bi-mode branch prediction when employing the segmented exclusive or G-share branch prediction scheme. Building on this foundation, the on-chip debugging system and memory management unit have been implemented. The processor reaches 1.389 Dhrystone/MHz and 2.802 Coremark/MHz.

Agile C-states: A Core C-state Architecture for Latency Critical Applications Optimizing both Transition and Cold-Start Latency

Latency critical applications running in modern datacenters exhibit irregular request arrival patterns and are implemented using multiple services with strict latency requirements (30 μs –250 μs ). These characteristics render existing energy saving idle CPU sleep states ineffective due to the performance overhead caused by the state’s transition latency. Besides the state transition latency, another important contributor to the performance overhead of sleep states is the cold-start latency, or in other words, the time required to warm-up microarchitectural state (e.g., cache contents, branch predictor metadata) that is flushed or discarded when transitioning to a lower-power state. Both the transition latency and cold-start latency can be particularly detrimental to the performance of latency critical applications with short execution times. While prior work focuses on mitigating the effects of transition and cold-start latency by optimizing request scheduling, in this work, we propose a redesign of the Core C-state architecture for latency-critical applications. In particular, we introduce C6Awarm a new Agile Core C-state that drastically reduces the performance overhead caused by idle sleep state transition latency and cold-start latency, while maintaining significant energy savings. C6Awarm achieves its goals by implementing 1) medium-grained power gating, 2) preserving the microarchitectural state of the core and 3) by keeping the clock generator and PLL active and locked. Our analysis for a set of microservices based on an Intel Skylake server, shows that C6Awarm manages to reduce the energy consumption by up to \(70\% \) with limited performance degradation (at-most \(2\% \) ).

A Hardware Security Protection Method for Conditional Branches of Embedded Systems.

The branch prediction units (BPUs) generally have security vulnerabilities, which can be used by attackers to tamper with the branches, and the existing protection methods cannot defend against these attacks. Therefore, this article proposes a hardware security protection method for conditional branches of embedded systems. This method calculates the number of branch target buffer (BTB) updates every 80 clock cycles. If the number exceeds the set threshold, the BTB will be locked and prevent any process from tampering with the BTB entries, thereby resisting branch prediction analysis (BPA) attacks. Moreover, to prevent attackers from stealing the critical information of branches, the method designs the hybrid arbiter physical unclonable function (APUF) circuit to encrypt and decrypt the directions, addresses, and indexes of branches. This circuit combines the advantages of double APUF and Feed-Forward APUF, which can enhance the randomness of output response and resist machine learning attacks. If attackers still successfully tamper with the branches and disrupt the control flow integrity (CFI), this method detects tampering with the instruction codes, jump addresses, and jump directions in a timely manner through dynamic and static label comparison. The proposed method is implemented and tested on FPGA. The experimental results show that this method can achieve fine-grained security protection for conditional branches, with about 5.4% resource overhead and less than 5.5% performance overhead.

Large Language Model Inference Acceleration Based on Hybrid Model Branch Prediction

As the size of deep learning models continues to expand, the elongation of inference time has gradually evolved into a significant challenge to efficiency and practicality for autoregressive models. This work introduces a hybrid model acceleration strategy based on branch prediction, which accelerates autoregressive model inference without requiring retraining and ensures output consistency with the original model. Specifically, the algorithm employs two models with different parameter sizes aimed at the same task. The smaller model generates a series of potential tokens that are then parallelly validated by the larger model to determine their acceptability. By orchestrating the workflow of the large and small models through a branch-prediction strategy, the algorithm conceals the validation time of the larger model when predictions are successful, thereby accelerating inference. We propose a binomial distribution-based prediction function that blends theoretical principles with empirical evidence, specifically designed for the nuanced requirements of accelerating inference within a hybrid model framework. The entire algorithm was designed and implemented on the llama model for text generation and translation tasks. The experimental results indicate significant improvements. The proposed algorithm achieves a 1.2× to 3.4× increase in inference speed compared to the original model, consistently outperforming the speculative sampling inference acceleration algorithm.

A comparative study of 11 non-linear regression models highlighting autoencoder, DBN, and SVR, enhanced by SHAP importance analysis in soybean branching prediction.

To explore a robust tool for advancing digital breeding practices through an artificial intelligence-driven phenotype prediction expert system, we undertook a thorough analysis of 11 non-linear regression models. Our investigation specifically emphasized the significance of Support Vector Regression (SVR) and SHapley Additive exPlanations (SHAP) in predicting soybean branching. By using branching data (phenotype) of 1918 soybean accessions and 42k SNP (Single Nucleotide Polymorphism) polymorphic data (genotype), this study systematically compared 11 non-linear regression AI models, including four deep learning models (DBN (deep belief network) regression, ANN (artificial neural network) regression, Autoencoders regression, and MLP (multilayer perceptron) regression) and seven machine learning models (e.g., SVR (support vector regression), XGBoost (eXtreme Gradient Boosting) regression, Random Forest regression, LightGBM regression, GPs (Gaussian processes) regression, Decision Tree regression, and Polynomial regression). After being evaluated by four valuation metrics: R2 (R-squared), MAE (Mean Absolute Error), MSE (Mean Squared Error), and MAPE (Mean Absolute Percentage Error), it was found that the SVR, Polynomial Regression, DBN, and Autoencoder outperformed other models and could obtain a better prediction accuracy when they were used for phenotype prediction. In the assessment of deep learning approaches, we exemplified the SVR model, conducting analyses on feature importance and gene ontology (GO) enrichment to provide comprehensive support. After comprehensively comparing four feature importance algorithms, no notable distinction was observed in the feature importance ranking scores across the four algorithms, namely Variable Ranking, Permutation, SHAP, and Correlation Matrix, but the SHAP value could provide rich information on genes with negative contributions, and SHAP importance was chosen for feature selection. The results of this study offer valuable insights into AI-mediated plant breeding, addressing challenges faced by traditional breeding programs. The method developed has broad applicability in phenotype prediction, minor QTL (quantitative trait loci) mining, and plant smart-breeding systems, contributing significantly to the advancement of AI-based breeding practices and transitioning from experience-based to data-based breeding.

SecurityCloak: Protection against cache timing and speculative memory access attacks

Microarchitectural innovations such as deep cache hierarchies, out-of-order execution, branch prediction and speculative execution in modern processors have made possible to meet ever-increasing demands for performance. However, these innovations have inadvertently introduced vulnerabilities that are exploited by cache-side channel attacks such as Flush & Reload, Prime & Probe, Evict & Time, and attacks such as Spectre and Meltdown that exploit speculative executions. These attacks can potentially leak information which should be secured.Mitigating the attacks while preserving the performance of out-of-order execution has been a challenge. Previous hardware mitigation techniques against cache timing or side-channel attacks include complex cache indexing mechanisms, encrypting addresses, partitioning cache memories, assigning specific ways of a set for each process, or obfuscating cache accesses by using ghost threads. Previous techniques for preventing or at least mitigating attacks based on speculative executions include hiding speculative data accesses using separate buffers or caches, or undoing the effects of speculation throughout program execution. Most techniques address either attacks that exploit speculation such as Spectre or cache side-channel attacks but not both. In many cases, changes to the microarchitecture with additional hardware are needed to implement the security protection. In some cases the mitigations cause performance penalties. In contrast we present very simple designs aimed at preventing both timing based cache side-channel attacks and Spectre style attacks based on speculative executions. Our approach combines obfuscation of cache timing making it more difficult for side-channel attacks to succeed and delaying speculative data accesses that miss in cache until the speculation is resolved. We will show that these approaches prevent both timing attacks such as Flush & Reload, Prime & Probe, Evict & Time as well as speculative attacks such as Spectre. Our technique requires very minimal changes to hardware.

Branch Predictor Design for Energy Harvesting Powered Nonvolatile Processors

Branch Predictor Design for Energy Harvesting Powered Nonvolatile Processors

Access Interval Prediction by Partial Matching for Tightly Coupled Memory Systems

In embedded systems, tightly coupled memories (TCMs) are usually shared between multiple masters for the purpose of hardware efficiency and software flexibility. On the one hand, memory sharing improves area utilization, but on the other hand, this can lead to a performance degradation due to an increase in access conflicts. To mitigate the associated performance penalty, access interval prediction (AIP) has been proposed. In a similar fashion to branch prediction, AIP exploits program flow regularity to predict the cycle of the next memory access. We show that this structural similarity allows for adaption of state-of-the-art branch predictors, such as Prediction by Partial Matching (PPM) and the TAgged GEometric history length (TAGE) branch predictor. Our analysis on memory access traces reveals that PPM predicts 99 percent of memory accesses. As PPM does not lend itself to hardware implementation, we also present the PPM-based TAGE access interval predictor which attains an accuracy of over 97 percent outperforming all previously presented implementable AIP schemes.

Flexible Dual-Branch Siamese Network: Learning Location Quality Estimation and Regression Distribution for Visual Tracking

Anchor-free based trackers introduce an extra branch in addition to classification and regression branches in the network to achieve comparable performance with anchor-based trackers. This extra branch is usually trained independently in the training phase and is used in combination with other branches in the inference phase. However, this can increase the inconsistency between the inference phase and the training phase, potentially degrading the tracking performance. To address this problem, we propose a new Siamese network-based object tracking framework that eliminates this inconsistency by unifying classification and additional branch tasks to achieve learning location quality estimation. Furthermore, regression tasks for bounding boxes are widely formulated based on Dirac <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\delta$</tex-math> </inline-formula> distribution. Though this assumption works well for many scenarios, it restricts the prediction of regression branches. To overcome this restriction, we propose discretizing the continuous offset of the regression branch into multiple offset predictions, which enables the network to learn more flexible distributions automatically. Meanwhile, the discrete distribution prediction of regression branches is utilized to further guide the classification of the trackers. Extensive experiments on the widely accepted benchmarks demonstrate the effectiveness and efficiency of the proposed model.

RenderBench: The CPU Rendering Benchmark Suite Based on Microarchitecture-Independent Characteristics

This research addresses the issue of evaluating CPU rendering performance by introducing the innovative benchmark test suite construction method RenderBench. This method combines CPU microarchitecture features with rendering task characteristics to comprehensively assess CPU performance across various rendering tasks. Adhering to principles of representativeness and comprehensiveness, the constructed benchmark test suite encompasses diverse rendering tasks and scenarios, ensuring accurate capture of CPU performance features. Through data sampling and in-depth analysis, this study focuses on the role of microarchitecture-independent features in rendering programs, including instruction-level parallelism, instruction mix, branch prediction capability, register dependency distance, data flow stride, and memory reuse distance. The research findings reveal significant variations in rendering programs across these features. For instance, in terms of instruction-level parallelism, rendering programs demonstrate a high level of ILP (instruction-level parallelism), with an average value of 5.70 for ILP256, surpassing benchmarks such as Mibench and NAS Parallel Benchmark. Furthermore, in aspects such as instruction mix, branch prediction capability, register dependency distance, data flow stride, and memory reuse distance, rendering programs exhibit distinct characteristics. Through the application of the RenderBench method, a scalable and highly representative benchmark test suite was constructed, facilitating an in-depth exploration of CPU performance bottlenecks in rendering tasks. By delving into microarchitecture-independent features, this study provides profound insights into rendering program performance, offering valuable guidance for optimizing CPU rendering performance. The application of ensemble learning models, such as random forest, XGBoost, and ExtraTrees, reveals the significant influence of features like floating-point computation, memory access patterns, and register usage on CPU rendering program performance. These insights not only offer robust guidance for performance optimization but also underscore the importance of feature selection and algorithm choice. In summary, the results of feature importance ranking in this study provide beneficial directions and deep insights for the optimization and enhancement of CPU rendering program performance. These findings are poised to exert a positive impact on future research and development endeavors.

Evaluation and Benefit of Imprecise Value Prediction for Certain Types of Instructions

Based on branch prediction, value prediction has emerged as a solution for problems caused by true data dependencies in pipelined processors. While branch predictors have binary outcomes (taken/not taken), value predictors face a challenging task as their outcomes can take any value. Because of that, coverage is reduced to enhance high accuracy and minimise costly recovery from misprediction. This paper evaluates value prediction, focusing on instruction execution with imprecisely predicted operands whose result can still be correct. Two analytical models are introduced to represent instruction execution with value prediction. One model focuses on correctly predicted operands, while the other allows for imprecisely predicted operands as long as the instruction results remain correct. A trace-driven simulator was developed for simulation purposes, implementing well-known predictors and some of the predictors presented at the latest Championship Value Prediction. The gem5 simulator was upgraded to generate program traces of SPEC and EEMBC benchmarks that were used in simulations. Based on the simulation result, proposed analytical models were compared to reveal the conditions under which a model with imprecisely predicted operands, but still correct results, achieved better execution time than a model with correctly predicted operands. Analysis revealed that the accuracy of the correct instruction result based on the predicted operand, even when the predicted operand is imprecise, is higher than the accuracy of the correctly predicted operand. The accuracy improvement ranges from 0.8% to 44%, depending on the specific predictor used.

Design of a High Performance Vector Processor Based on RISIC-V Architecture

This paper proposes a high performance Vector processor based on the high performance Embedded Core which is named TS800. The TS800 is a 4-core processor based on RISC-V architecture, implements IMAFDV instruction set, supports L2 Cache, branch prediction, sequential pipeline, and dual-issue structure. The traditional CPU mainly supports Scalar calculations, or only supports Vector calculations. For applications such as image and signal processing, there are a large number of data parallel computing operations. To solve the problem of low performance of parallel data calculations in industrial power applications, it is proposed to add VPU hardware implementation in the TS800. The TS800 can support FFT algorithm, adaptive controllers Reinforcement learning and learning-based underlying algorithm requirements. In this paper, the module and data flow between each processing unit and the control circuit, that is, the hardware realization of VPU module are proposed. Large-area units such as float arithmetic, multiplication and division are multiplexed with the Scalar operator in the CPU, while the control circuit is placed in the VPU-ALU, and the area is small. Units such as arithmetic and logic operation instructions, shift operation instructions, comparison operation instructions, and permutation instruction are implemented through the VPU-ALU, which makes the overall design area smaller and the performance better. At the same time, through the fir, fft, conv, matrix, Signal Converge and variance test, it is proved that while executing the same program, the running time of the cpu only with Scalar is 1.44 to 9.55 times that of the CPU with Vector module, which can support the underlying algorithm of the adaptive controller.

Modified Architecture of Tomasulo’s Algorithm using Adaptive Branch Predictor

This paper proposes modified architecture to implement the Tomasulo’s Algorithm using Adaptive branch prediction scheme, which evaluate the branch outcome by taking both local and global history dynamically. The choice of local vs. global branch prediction is made dynamically on a path based predictor that decides which branch must be taken on past correctness of choice.

A conditional branch predictor based on weightless neural networks

Conditional branch prediction allows the speculative fetching and execution of instructions before knowing the direction of conditional statements. As in other areas, machine learning techniques are a promising approach to building branch predictors, e.g., the Perceptron predictor. However, those traditional solutions demand large input sizes, which impose a considerable area overhead. We propose a conditional branch predictor based on the WiSARD (Wilkie, Stoneham, and Aleksander’s Recognition Device) weightless neural network model. The WiSARD-based predictor implements one-shot online training designed to address branch prediction as a binary classification problem. We compare the WiSARD-based predictor with two state-of-the-art predictors: TAGE-SC-L (TAgged GEometric-Statistical Corrector-Loop) and the Multiperspective Perceptron. Our experimental evaluation shows that our proposed predictor, with a smaller input size, outperforms the perceptron-based predictor by about 0.09% and achieves similar accuracy to that of TAGE-SC-L. In addition, we perform an experimental sensitivity analysis to find the best predictor for each dataset, and based on these results, we designed new specialized predictors using a particular parameter composition for each dataset. The results show that the specialized WiSARD-based predictor outperforms the state-of-the-art by more than 2.3% in the best case. Furthermore, through the implementation of specialized predictor classifiers, we discovered that utilizing 90% of the specialized predictor for a specific dataset yielded comparable performance to the corresponding specialized predictor.

An Experimental Approach to Estimation of the Energy Cost of Dynamic Branch Prediction in an Intel High-Performance Processor

Power and energy efficiency are among the most crucial requirements in high-performance and other computing platforms. In this work, extensive experimental methods and procedures were used to assess the power and energy efficiency of fundamental hardware building blocks inside a typical high-performance CPU, focusing on the dynamic branch predictor (DBP). The investigation relied on the Running Average Power Limit (RAPL) interface from Intel, a software tool for credibly reporting the power and energy based on instrumentation inside the CPU. We used well-known microbenchmarks under various run conditions to explore potential pitfalls and to develop precautions to raise the precision of the measurements obtained from RAPL for more reliable power estimation. The authors discuss the factors that affect the measurements and share the difficulties encountered and the lessons learned.

A Dynamic and Static Binary Translation Method Based on Branch Prediction

Binary translation is an important technique for achieving cross-architecture software migration. However, mainstream dynamic binary translation frameworks, such as QEMU, often generate a large amount of redundant code, which degrades the efficiency of the target code. To this end, we propose a dynamic–static binary translation method based on branch prediction. It first identifies parts of translation blocks following static branch prediction techniques. Then it translates these translation blocks into less-redundant native code blocks by canonical static translation algorithms. Finally, it executes all code blocks that are translated either statically or dynamically by correctly maintaining and switching their running contexts. In order to correctly weave the two types of translation activities, the proposed method only translates the next translation block that is data-independent from the current one by the active variable analysis algorithm, and records and shares the intermediate states of the dynamic and static translation activities via a carefully designed data structure. In particular, a shadow register-based context recovery mechanism is proposed to correctly record the running context of static translation blocks, and to correctly recover the context for dynamically translating and running blocks that were not statically translated. We also designed an adaptive memory optimization mechanism to dynamically release the memory of the mispredicted translation blocks. We implemented a dynamic–static binary translation framework by extending QEMU, called BP-QEMU (QEMU with branch prediction). We evaluated the translation correctness of BP-QEMU using the testing programs for the ARM and PPC instruction sets from QEMU, and evaluated the performance of BP-QEMU using the CoreMark benchmark code. The experimental results show that BP-QEMU can translate the instructions from the ARM and PPC architectures correctly; moreover, the average execution efficiency of the CoreMark code on BP-QEMU improves by 13.3% compared to that of QEMU.

Online Code Layout Optimizations via OCOLOS

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">The processor front-end has become an increasingly important bottleneck in recent years due to growing application code footprints, particularly in data centers. Profile-guided optimizations performed by compilers represent a promising approach, as they rearrange code to maximize instruction cache locality and branch prediction efficiency along a relatively small number of hot code paths. However, these optimizations require continuous profiling and rebuilding of applications to ensure that the code layout matches the collected profiles. In this paper, we propose OCOLOS, the first online code layout optimization system for unmodified applications written in unmanaged languages. OCOLOS allows profile-guided optimization to be performed on a running process, instead of being performed offline and requiring the application to be re-launched. Our experiments show that OCOLOS can accelerate MySQL by up to 41%.</i>