Tail Latency Research Articles

Ubik

Chip-multiprocessors (CMPs) must often execute workload mixes with different performance requirements. On one hand, user-facing, latency-critical applications (e.g., web search) need low tail (i.e., worst-case) latencies, often in the millisecond range, and have inherently low utilization. On the other hand, compute-intensive batch applications (e.g., MapReduce) only need high long-term average performance. In current CMPs, latency-critical and batch applications cannot run concurrently due to interference on shared resources. Unfortunately, prior work on quality of service (QoS) in CMPs has focused on guaranteeing average performance, not tail latency. In this work, we analyze several latency-critical workloads, and show that guaranteeing average performance is insufficient to maintain low tail latency, because microarchitectural resources with state, such as caches or cores, exert inertia on instantaneous workload performance. Last-level caches impart the highest inertia, as workloads take tens of milliseconds to warm them up. When left unmanaged, or when managed with conventional QoS frameworks, shared last-level caches degrade tail latency significantly. Instead, we propose Ubik, a dynamic partitioning technique that predicts and exploits the transient behavior of latency-critical workloads to maintain their tail latency while maximizing the cache space available to batch applications. Using extensive simulations, we show that, while conventional QoS frameworks degrade tail latency by up to 2.3x, Ubik simultaneously maintains the tail latency of latency-critical workloads and significantly improves the performance of batch applications.

Ubik

Chip-multiprocessors (CMPs) must often execute workload mixes with different performance requirements. On one hand, user-facing, latency-critical applications (e.g., web search) need low tail (i.e., worst-case) latencies, often in the millisecond range, and have inherently low utilization. On the other hand, compute-intensive batch applications (e.g., MapReduce) only need high long-term average performance. In current CMPs, latency-critical and batch applications cannot run concurrently due to interference on shared resources. Unfortunately, prior work on quality of service (QoS) in CMPs has focused on guaranteeing average performance, not tail latency. In this work, we analyze several latency-critical workloads, and show that guaranteeing average performance is insufficient to maintain low tail latency, because microarchitectural resources with state, such as caches or cores, exert inertia on instantaneous workload performance. Last-level caches impart the highest inertia, as workloads take tens of milliseconds to warm them up. When left unmanaged, or when managed with conventional QoS frameworks, shared last-level caches degrade tail latency significantly. Instead, we propose Ubik, a dynamic partitioning technique that predicts and exploits the transient behavior of latency-critical workloads to maintain their tail latency while maximizing the cache space available to batch applications. Using extensive simulations, we show that, while conventional QoS frameworks degrade tail latency by up to 2.3x, Ubik simultaneously maintains the tail latency of latency-critical workloads and significantly improves the performance of batch applications.

Speeding up distributed request-response workflows

We found that interactive services at Bing have highly variable datacenter-side processing latencies because their processing consists of many sequential stages, parallelization across 10s-1000s of servers and aggregation of responses across the network. To improve the tail latency of such services, we use a few building blocks: reissuing laggards elsewhere in the cluster, new policies to return incomplete results and speeding up laggards by giving them more resources. Combining these building blocks to reduce the overall latency is non-trivial because for the same amount of resource (e.g., number of reissues), different stages improve their latency by different amounts. We present Kwiken, a framework that takes an end-to-end view of latency improvements and costs. It decomposes the problem of minimizing latency over a general processing DAG into a manageable optimization over individual stages. Through simulations with production traces, we show sizable gains; the 99th percentile of latency improves by over 50% when just 0.1% of the responses are allowed to have partial results and by over 40% for 25% of the services when just 5% extra resources are used for reissues.

Strategies to target kyotorphin analogues to the brain.

The design, synthesis, and pharmacological evaluation of brain-targeted chemical delivery systems (CDS) for a kyotorphin analogue (Tyr-Lys) are described. The brain-targeted compound contains the active peptide in a packaged, disguised form, flanked between the lipophilic cholesteryl ester on the C-terminus and the 1, 4-dihydrotrigonellyl redox targetor, attached to the N-terminus through strategically selected L-amino acid(s) spacer. It was found that for successful brain targeting, the epsilon-amine of Lys needs to be also converted to a lipophilic function. Through sequential enzymatic bioactivation, the Tyr-Lys dipeptide is released in a sustained manner, producing significant and prolonged analgesic activity as demonstrated by the rat tail latency test. An alternate strategy was also employed. Lys was replaced by a redox amino acid pair, Nys+ left and right arrow Nys, the nicotinamide left and right arrow 1,4-dihydronicotinamide analogues of Lys (Nys+ is 2-amino-6-(3-carbamoyl-1-pyridiniumyl)hexanoic acid). The Nys form is lipophilic and facilitates delivery in addition to the C- and N-terminal lipophilic functions. Enzymatic oxidation to Nys+ provides the lock-in, followed by removal of the lipophilic groups, releasing Tyr-Nys+ from the brain-targeted analogue (BTRA). Nys+ was shown to be an effective substitution for Arg or Lys. The activities of the CDS and BTRA, respectively, were antagonized by naloxone, supporting the designed brain-targeted processes. The most potent compound is the two-proline spacer containing CDS (CDS-PP), followed by the BTRA.

Cardiovascular, Respiratory, and Analgesic Effects of Fentanyl in Unanesthetized Rhesus Monkeys

To determine the suitability of the rhesus monkey as a model for investigation of opioids, we examined the analgesic, respiratory, and cardiovascular effects of fentanyl in six adult male rhesus monkeys. Fentanyl was administered in sequential bolus injections of 2, 4, 16, 64, and 128 micrograms/kg, with 10 min between each dose. Arterial plasma fentanyl concentrations and blood gas tensions were measured 3 and 9 min after each dose and 1, 2, 5, 20, 60, and 120 min after the final dose. At the same time periods, mean systemic arterial, pulmonary arterial, central venous, and pulmonary capillary wedge pressures, cardiac output, heart rate, and respiratory rate were measured. Analgesia was quantified as the time required for tail withdrawal from a standardized noxious stimulus. Tail latency response time increased significantly after the 4-microgram/kg dose (plasma fentanyl concentration = 2.7 +/- 0.9 ng/mL). Maximum tail latency response time was attained after the 64-micrograms/kg dose (43.4 +/- 26.0 ng/mL). Respiratory rate decreased significantly after the 2-microgram/kg dose, and PaCO2 increased significantly after the 4-microgram/kg dose. All animals became apneic, requiring tracheal intubation and controlled ventilation, after the 64-micrograms/kg dose. Also, mean arterial pressure and cardiac output decreased significantly after the 64-micrograms/kg dose. There were no other significant cardiovascular changes. Peak plasma fentanyl concentration after the 128-micrograms/kg dose was 117.0 +/- 49.6 ng/mL. It appears that plasma concentrations of approximately 40 ng/mL are sufficient to reach the full cardiovascular, respiratory, and analgesic effects of fentanyl in the rhesus monkey. Significant respiratory and analgesic effects are evident at concentrations as low as 3 ng/mL.(ABSTRACT TRUNCATED AT 250 WORDS)