Latency Of Operations Research Articles

Speculative hardware/software co-designed floating-point multiply-add fusion

A Fused Multiply-Add (FMA) instruction is currently available in many general-purpose processors. It increases performance by reducing latency of dependent operations and increases precision by computing the result as an indivisible operation with no intermediate rounding. However, since the arithmetic behavior of a single-rounding FMA operation is different than independent FP multiply followed by FP add instructions, some algorithms require significant revalidation and rewriting efforts to work as expected when they are compiled to operate with FMA--a cost that developers may not be willing to pay. Because of that, abundant legacy applications are not able to utilize FMA instructions. In this paper we propose a novel HW/SW collaborative technique that is able to efficiently execute workloads with increased utilization of FMA, by adding the option to get the same numerical result as separate FP multiply and FP add pairs. In particular, we extended the host ISA of a HW/SW co-designed processor with a new Combined Multiply-Add (CMA) instruction that performs an FMA operation with an intermediate rounding. This new instruction is used by a transparent dynamic translation software layer that uses a speculative instruction-fusion optimization to transform FP multiply and FP add sequences into CMA instructions. The FMA unit has been slightly modified to support both single-rounding and double-rounding fused instructions without increasing their latency and to provide a conservative fall-back path in case of mispeculation. Evaluation on a cycle-accurate timing simulator showed that CMA improved SPECfp performance by 6.3% and reduced executed instructions by 4.7%.

Speculative hardware/software co-designed floating-point multiply-add fusion

A Fused Multiply-Add (FMA) instruction is currently available in many general-purpose processors. It increases performance by reducing latency of dependent operations and increases precision by computing the result as an indivisible operation with no intermediate rounding. However, since the arithmetic behavior of a single-rounding FMA operation is different than independent FP multiply followed by FP add instructions, some algorithms require significant revalidation and rewriting efforts to work as expected when they are compiled to operate with FMA--a cost that developers may not be willing to pay. Because of that, abundant legacy applications are not able to utilize FMA instructions. In this paper we propose a novel HW/SW collaborative technique that is able to efficiently execute workloads with increased utilization of FMA, by adding the option to get the same numerical result as separate FP multiply and FP add pairs. In particular, we extended the host ISA of a HW/SW co-designed processor with a new Combined Multiply-Add (CMA) instruction that performs an FMA operation with an intermediate rounding. This new instruction is used by a transparent dynamic translation software layer that uses a speculative instruction-fusion optimization to transform FP multiply and FP add sequences into CMA instructions. The FMA unit has been slightly modified to support both single-rounding and double-rounding fused instructions without increasing their latency and to provide a conservative fall-back path in case of mispeculation. Evaluation on a cycle-accurate timing simulator showed that CMA improved SPECfp performance by 6.3% and reduced executed instructions by 4.7%.

Novel flat datacenter network architecture based on scalable and flow-controlled optical switch system

We propose and demonstrate an optical flat datacenter network based on scalable optical switch system with optical flow control. Modular structure with distributed control results in port-count independent optical switch reconfiguration time. RF tone in-band labeling technique allowing parallel processing of the label bits ensures the low latency operation regardless of the switch port-count. Hardware flow control is conducted at optical level by re-using the label wavelength without occupying extra bandwidth, space, and network resources which further improves the performance of latency within a simple structure. Dynamic switching including multicasting operation is validated for a 4 x 4 system. Error free operation of 40 Gb/s data packets has been achieved with only 1 dB penalty. The system could handle an input load up to 0.5 providing a packet loss lower that 10(-5) and an average latency less that 500 ns when a buffer size of 16 packets is employed. Investigation on scalability also indicates that the proposed system could potentially scale up to large port count with limited power penalty.

Design of a universal logic block for fault-tolerant realization of any logic operation in trapped-ion quantum circuits

This paper presents a physical mapping tool for quantum circuits, which generates the optimal Universal Logic Block (ULB) that can perform any logical fault-tolerant (FT) quantum operations with the minimum latency. The operation scheduling, placement, and qubit routing problems tackled by the quantum physical mapper are highly dependent on one another. More precisely, the scheduling solution affects the quality of the achievable placement solution due to resource pressures that may be created as a result of operation scheduling whereas the operation placement and qubit routing solutions influence the scheduling solution due to resulting distances between predecessor and current operations, which in turn determines routing latencies. The proposed flow for the quantum physical mapper captures these dependencies by applying (i) a loose scheduling step, which transforms an initial quantum data flow graph into one that explicitly captures the no-cloning theorem of the quantum computing and then performs instruction scheduling based on a modified force-directed scheduling approach to minimize the resource contention and quantum circuit latency, (ii) a placement step, which uses timing-driven instruction placement to minimize the approximate routing latencies while making iterative calls to the aforesaid force-directed scheduler to correct scheduling levels of quantum operations as needed, and (iii) a routing step that finds dynamic values of routing latencies for the qubits. In addition to the quantum physical mapper, an approach is presented to determine the single best ULB size for a target quantum circuit by examining the latency of different FT quantum operations mapped onto different ULB sizes and using information about the occurrence frequency of operations on critical paths of the target quantum algorithm to weigh these latencies.

Reliable Content Distribution in P2P Networks Based on Peer Groups

Peer-to-Peer (P2P) networks are highly dynamic systems which are very popular for content distribution in the Internet. A single peer remains in the system for an unpredictable amount of time, and the rate in which peers enter and leave the system, i.e. the churn, is often high. A user that is obtaining content from a selected peer is frequently informed that particular peer is not available anymore, and is asked to select another peer, or will have another peer assigned, often without enough checks to confirm that the content provided by the new peer presents the same quality of the previous peer. In this work we present a strategy based on group communication for transparent and robust content access in P2P networks. Instead of accessing a single peer for obtaining the desired content, a user request is received and processed by a group of peers. This group of peers, called PCG (Peer Content Group) provides reliable content access in sense that even as members of the group crash or leave the system, users continue to receive the content if at least one group member remains fault-free. Each PCG member is capable of independently serving the request. A PCG is transparent to the user, as the group interface is identical to the interface provided by a single peer. A group member is elected to serve each request. A fault monitoring component allows the detection of member crashes. If the peer is serving request crashes, another group member is elected to continue providing the service. The PCG and a P2P file sharing applications were implemented in the JXTA platform. Evaluation results are presented showing the latency of group operations and system components.

Optimizing Wait States in the Synthesis of Memory References with Unpredictable Latencies

We consider the problem of synthesizing circuits (from C to Verilog) that are optimized to handle unpredictable latencies of memory operations. Unpredictable memory latencies can occur due to the use of on chip caches, DRAM memory modules, buffers/queues, or multiport memories. Typically, high-level synthesis compilers assume fixed and known memory latencies, and thus are able to schedule the code’s operations efficiently. The operations in the source code are scheduled into states of a state machine whose states will be synthesized to Verilog. The goal is to minimize scheduling length by maximizing the number of operations (and in particular memory operations) that are executed in parallel at the same state. However, with unpredictable latencies, there can be an exponential number of possible orders in which these parallel memory operations can terminate. Thus, in order to minimize the scheduling, we need a different schedule for any such order. This is not practical, and we show a technique of synthesizing a compact state machine that schedules only a small subset of these possible termination orders. Our results show that this compact state machine can improve the execution time compared to a regular scheduling that waits for the termination of all the active memory references in every state.

Improving branch divergence performance on GPGPU with a new PDOM stack and multi-level warp scheduling

Abstract General-purpose graphics processing unit (GPGPU) plays an important role in massive parallel computing nowadays. A GPGPU core typically holds thousands of threads, where hardware threads are organized into warps. With the single instruction multiple thread (SIMT) pipeline, GPGPU can achieve high performance. But threads taking different branches in the same warp violate SIMD style and cause branch divergence. To support this, a hardware stack is used to sequentially execute all branches. Hence branch divergence leads to performance degradation. This article represents the PDOM (post dominator) stack as a binary tree, and each leaf corresponds to a branch target. We propose a new PDOM stack called PDOM-ASI, which can schedule all the tree leaves. The new stack can hide more long operation latencies with more schedulable warps without the problem of warp over-subdivision. Besides, a multi-level warp scheduling policy is proposed, which lets part of the warps run ahead and creates more opportunities to hide the latencies. The simulation results show that our policies achieve 10.5% performance improvements over baseline policies with only 1.33% hardware area overhead.

Virtualizing ARM VFP (Vector Floating-Point) with Xen-ARM

VFP is a Vector Floating-Point unit in ARM processors. It enables ARM processors to handle auxiliary floating-point operations with hardware, which has become an essential part for performance in recent mobile devices. However, mobile virtualization such as Xen-ARM does not support virtual VFP, so floating point operations are very slow in mobile virtual machine. To overcome the performance limitation of floating-point operations in Xen-ARM virtualization, this paper presents a new virtual VFP so that applications can take the advantage of VFP hardware. With our virtual VFP, Xen-Linux reduces floating point operation latency up to one eighth from the existing software emulation version. In addition, result from mibench with virtual VFP presents 3-4 times higher throughput than that from software emulation. Furthermore, virtual VFP reduces code sizes and enhances power-efficiency.

Quantifying eventual consistency with PBS

Data store replication results in a fundamental trade-off between operation latency and data consistency. At the weak end of the consistency spectrum is eventual consistency providing no limit to the staleness of data returned. However, anecdotally, eventual consistency is often “good enough” for practitioners given its latency and availability benefits. In this work, we explain why eventually consistent systems are regularly acceptable in practice, analyzing both the staleness of data they return and the latency benefits they offer. We introduce Probabilistically Bounded Staleness (PBS), a consistency model which provides expected bounds on data staleness with respect to both versions and wall clock time. We derive a closed-form solution for versioned staleness as well as model real-time staleness under Internet-scale production workloads for a large class of quorum-replicated, Dynamo-style stores. Using PBS, we measure the latency–consistency trade-off for partial, non-overlapping quorum systems, including limited multi-object operations. We quantitatively demonstrate how and why eventually consistent systems frequently return consistent data within tens of milliseconds while offering significant latency benefits.

High-throughput low-latency fine-grained disk logging

Synchronously logging updates to persistent storage first and then asynchronously committing these updates to their rightful storage locations is a well-known and heavily used technique to improve the sustained throughput of write-intensive disk-based data processing systems, whose latency and throughput accordingly are largely determined by the latency and throughput of the underlying logging mechanism. The conventional wisdom is that logging operations are relatively straightforward to optimize because the associated disk access pattern is largely sequential. However, it turns out that to achieve both high throughput and low latency for fine-grained logging operations, whose payload size is smaller than a disk sector, is extremely challenging. This paper describes the experiences and lessons we have gained from building a disk logging system that can successfully deliver over 1.2 million 256-byte logging operations per second, with the average logging latency below 1 msec.

All-Optical Header Recognition and Packet Switching Using Polarization Bistable VCSEL

We demonstrate an optical header processing system using a 1.55-μm polarization bistable vertical-cavity surface-emitting laser (VCSEL) that operates as an all-optical AND gate and a holding element. Depending on the state of 1 bit in the 4-bit header, the VCSEL held one of two-orthogonal polarization states. The light of the VCSEL output through a polarizer switched and held the output state of an optical switch. This enables the payload to be sent to the destination determined by the header. The characteristics of the polarization bistable VCSEL enable low-power consumption and lower latency operation. The system also has the potential to process multibit optical headers involved in ultrafast optical packets.

Decimal SRT Square Root: Algorithm and Architecture

Given the popularity of decimal arithmetic, hardware implementation of decimal operations has been a hot topic of research in recent decades. Besides the four basic operations, the square root can be implemented as an instruction directly in the hardware, which improves the performance of the decimal floating-point unit in the processors. Hardware implementation of decimal square rooters is usually done using either functional or digit-recurrence algorithms. Functional algorithms, entailing multiplication per iteration, seem inadequate to use for decimal square roots, given the high cost of decimal multipliers. On the other hand, digit-recurrence square root algorithms, particularly SRT (this method is named after its creators, Sweeney, Robertson, and Tocher) algorithms, are simple and well suited for decimal arithmetic. This paper, with the intention of reducing the latency of the decimal square root operation while maintaining a reasonable cost, proposes an SRT algorithm and the corresponding hardware architecture to compute the decimal square root. The proposed fixed-point square root design requires n+3 cycles to compute an n-digit root; the synthesis results show an area cost of about 31K NAND2 and a cycle time of 40 FO4. These results reveal the 14 % speed advantage of the proposed decimal square root architecture over the fastest previous work (which uses a functional algorithm) with about a quarter of the area.

Performance modeling and analysis of parallel Gaussian elimination on multi-core computers

Gaussian elimination is used in many applications and in particular in the solution of systems of linear equations. This paper presents mathematical performance models and analysis of four parallel Gaussian Elimination methods (precisely the Original method and the new Meet in the Middle –MiM– algorithms and their variants with SIMD vectorization) on multi-core systems. Analytical performance models of the four methods are formulated and presented followed by evaluations of these models with modern multi-core systems’ operation latencies. Our results reveal that the four methods generally exhibit good performance scaling with increasing matrix size and number of cores. SIMD vectorization only makes a large difference in performance for low number of cores. For a large matrix size (n⩾16K), the performance difference between the MiM and Original methods falls from 16× with four cores to 4× with 16K cores. The efficiencies of all four methods are low with 1K cores or more stressing a major problem of multi-core systems where the network-on-chip and memory latencies are too high in relation to basic arithmetic operations. Thus Gaussian Elimination can greatly benefit from the resources of multi-core systems, but higher performance gains can be achieved if multi-core systems can be designed with lower memory operation, synchronization, and interconnect communication latencies, requirements of utmost importance and challenge in the exascale computing age.

FPGA based High Speed Double Precision Floating Point Divider

Floating point arithmetic is widely used in many areas, especially scientific computation and signal processing. For many signal processing, and graphics applications, it is acceptable to trade off some accuracy (in the least significant bit positions) for faster and better implementations. Division is the third basic operation of arithmetic. However, most of these modern applications need higher frequency or low latency of operations with minimal area occupancy. In this paper we describe an implementation of high speed IEEE 754 double precision floating point divider using digit recurrence algorithm and targeted for Xilinx Virtex-6 Field Programmable Gate Array. Verilog is used to implement the design. The implemented design achieves 344.89 MFlops and this design occupies 653 slices. It handles the overflow, underflow and rounding mode.

FPGA based Implementation of High Speed Double Precision Floating Point Multiplier with Tiling Technique using Verilog

Floating point arithmetic is widely used in many areas, especially scientific computation and signal processing. For many signal processing, and graphics applications, it is acceptable to trade off some accuracy (in the least significant bit positions) for faster and better implementations. However, most of these modern applications need higher frequency or low latency of operations with minimal area occupancy. In this paper we describe an implementation of high speed IEEE 754 double precision floating point multiplier using tiling technique and targeted for Xilinx Virtex-6 Field Programmable Gate Array. Verilog is used to implement the design. The design achieved 436.815 MFlops with latency of seven clock cycles which is 97% fast compared to Xilinx floating point multiplier core. It handles the overflow, underflow cases and truncation rounding mode.

An asynchronous finite-state-machine-based buck-boost converter for on-chip adaptive power supply

In this paper we present an asynchronous finite-state machine digital controller co-integrated with an on-chip non-inverting buck-boost power converter with dynamic signal-tracking capabilities. The mostly-digital controller functionally implements a non-PWM zone-wise control law through asynchronous circuitry, thus exhibiting self-timed minimum latency and ultra low power operation due to gate switching activity. Experimental results on a 0.35 μm CMOS technology demonstrate an efficiency up to 80 % with a switching frequency of 2.86 MHz.

Exploring Locality of Reference in P2P VoD Systems

<?Pub Dtl=""?> A critical problem to peer-to-peer video-on-demand (P2P VoD) systems is to provide efficient user interactivity support. In this paper, we study intra- and inter-video operations separately, aiming to exploit the locality of reference in user access patterns and reduce the latency of these VoD operations. We first introduce the concepts of available, request and delivered locality in intra-video user access patterns and prove that high available locality exists in different videos by both simulation and theoretical analysis. Moreover, with a relaxed definition of data chunk holder, intra-video locality can facilitate a high likelihood of a peer seeking within a video, finding a holder of the requested data among its neighbors. Exploiting this property, an aggressive cached publish scheme is designed to build shortcuts over the DHT network so as to reduce the lookup delay. This scheme may be simple but it is practical and easy to implement. Inter-video locality is exploited via learning association rules from the collective viewing history. A fast association rule learning algorithm is proposed to infer the relations between videos in a distributed manner based on partial knowledge. Both search and content prefetch are incorporated to achieve low inter-video jump delay with minimal overhead. Our simulations demonstrate that the proposed schemes can reduce the buffer and lookup delay for seeking within a video and provide an efficient prediction-based prefetch scheme for inter-video access.

Probabilistically bounded staleness for practical partial quorums

Data store replication results in a fundamental trade-off between operation latency and data consistency. In this paper, we examine this trade-off in the context of quorum-replicated data stores. Under partial, or non-strict quorum replication, a data store waits for responses from a subset of replicas before answering a query, without guaranteeing that read and write replica sets intersect. As deployed in practice, these configurations provide only basic eventual consistency guarantees, with no limit to the recency of data returned. However, anecdotally, partial quorums are often "good enough" for practitioners given their latency benefits. In this work, we explain why partial quorums are regularly acceptable in practice, analyzing both the staleness of data they return and the latency benefits they offer. We introduce Probabilistically Bounded Staleness (PBS) consistency, which provides expected bounds on staleness with respect to both versions and wall clock time. We derive a closed-form solution for versioned staleness as well as model real-time staleness for representative Dynamo-style systems under internet-scale production workloads. Using PBS, we measure the latency-consistency trade-off for partial quorum systems. We quantitatively demonstrate how eventually consistent systems frequently return consistent data within tens of milliseconds while offering significant latency benefits.

Attribute-based Proxy Re-encryption with a Constant Number of Pairing Operations

Attribute-based encryption (ABE) is an encryption scheme in which the user is able to decrypt a ciphertext with associated attributes. However, the scheme does not offer the capability of decryption to others when the user is offline. For this reason, the attribute-based proxy re-encryption (ABPRE) scheme was proposed, which combines traditional proxy re-encryption with ABE, so a user is able to empower designated users to decrypt the re-encrypted ciphertext with the associated attributes of designated users. However, previous ABPRE schemes demands a number of pairing operations that imply huge computational overhead. To reduce the number of pairing operations, we reduce the pairing operations with exponent operations. This paper provides a novel approach to an ABPRE scheme with constant pairing operation latency.

Imagining the Image: Photography, psychoanalysis and the affects of latency

This essay brings together the figure of the photographer with that of the psychoanalyst by tracing their work with and through latency. Both photographer and psychoanalyst expose something latent in their work: the photographer through dealing with images that are yet to be revealed; the analyst through dealing with signs that are yet to be brought into a relief of signification. Appearing in either field, the latent finds its conceptual bearings in different contexts and by different means. This essay contends that latency forms a transdisciplinary modality, running across fields that stage its effects in dissimilar ways. By stepping outside the need to establish something analogous in latency's operation across the fields of photography and psychoanalysis, we discover what is proper to photography: the play of certainty and doubt around the production of images.