Context Switching Research Articles

Task Parallelism-Aware Deep Neural Network Scheduling on Multiple Hybrid Memory Cube-Based Processing-in-Memory

Processing-in-memory (PIM) comprises computational logic in the memory domain. It is the most promising solution to alleviate the memory bandwidth problem in deep neural network (DNN) processing. The hybrid memory cube (HMC), a 3D stacked memory structure, can efficiently implement the PIM architecture by maximizing the existing legacy hardware. To accelerate DNN inference, multiple HMCs can be connected, and data-independent tasks can be assigned to processing elements (PEs) within each HMC. However, owing to the packet-switched network structure, inter-HMC interconnects exhibit variable and unpredictable latencies depending on the data transmission path and link contention. A well-designed task schedule using context switching can effectively hide communication latency and improve PE utilization. Nevertheless, as the number of HMC increases, the variability of a wide range of inter-HMC communication latencies causes frequent context switching, degrading overall performance. This paper proposes a DNN task scheduling that can effectively utilize task parallelism by reducing the communication latency variance owing to HMC interconnect characteristics. Task partitions are generated to exploit parallelism while providing inter-HMC traffic within the sustainable link bandwidth. Task-to-HMC mapping is performed to hide the average communication latency of intermediate DNN processing results. A task schedule is generated using retiming to accelerate DNN inference while maximizing resource utilization. The effectiveness of the proposed method was verified through simulations using various realistic DNN applications performed on a ZSim x86-64 simulator. The simulations revealed that DNN processing with the proposed scheduling improved the DNN processing speed by reducing the processing time by 18.19% over conventional methods where each HMC operated independently.

An Adjustable Variant of Round Robin Algorithm Based on Clustering Technique

CPU scheduling is the basic task within any time-shared operating system. One of the main goals of the researchers interested in CPU scheduling is minimizing time cost. Comparing between CPU scheduling algorithms is subject to some scheduling criteria (e.g., turnaround time, waiting time and number of context switches (NCS)). Scheduling policy is divided into preemptive and non-preemptive. Round Robin (RR) algorithm is the most common preemptive scheduling algorithm used in the time-shared operating systems. In this paper, the authors proposed a modified version of the RR algorithm, called dynamic time slice (DTS), to combine the advantageous of the low scheduling overhead of the RR and favor short process for the sake of minimizing time cost. Each process has a weight proportional to the weights of all processes. The process’s weight determines its time slice within the current period. The authors benefit from the clustering technique in grouping the processes that are similar in their attributes (e.g., CPU service time, weight, allowed time slice (ATS), proportional burst time (PBT) and NCS). Each process in a cluster is assigned the average of the processes’ time slices in this cluster. A comparative study of six popular scheduling algorithms and the proposed approach on nine groups of processes vary in their attributes was performed and the evaluation was measured in terms of waiting and turnaround times, and NCS. The experiments showed that the proposed algorithm gives better results.

Interprocedural Context-Unbounded Program Analysis Using Observation Sequences

A classical result by Ramalingam about synchronization-sensitive interprocedural program analysis implies that reachability for concurrent threads running recursive procedures is undecidable. A technique proposed by Qadeer and Rehof, to bound the number of context switches allowed between the threads, leads to an incomplete solution that is, however, believed to catch “most bugs” in practice, as errors tend to occur within few contexts. The question of whether the technique can also prove the absence of bugs at least in some cases has remained largely open. Toward closing this gap, we introduce in this article the generic verification paradigm of observation sequences for resource-parameterized programs. Such a sequence observes how increasing the resource parameter affects the reachability of states satisfying a given property. The goal is to show that increases beyond some “cutoff” parameter value have no impact on the reachability—the sequence has converged . This allows us to conclude that the property holds for all parameter values. We applied this paradigm to the context- unbounded program analysis problem, choosing the resource to be the number of permitted thread context switches. The result is a partially correct interprocedural reachability analysis technique for concurrent shared-memory programs. Our technique may not terminate but is able to both refute and prove context-unbounded safety for such programs. We demonstrate the effectiveness and efficiency of the technique using a variety of benchmark programs. The safe instances cannot be proved safe by earlier, context-bounded methods.

FastUDP: a highly scalable user-level UDP framework in multi-core systems for fast packet I/O

Nowadays, many applications, e.g., network routers, distributed data process engines, firewall, need to transfer packets at linear rate. With the increasing data volume, the performance of cluster in data center is suffering increasingly severe congestion problem of massive message packets. Constructing a high-performance stream methodology of massive small message packets is fundamentally challenging. Although many works have been proposed to address the shortcomings, inefficiency of sending massive small packets via UDP protocol in traditional Linux kernel implementation is persisting, which includes high overhead from socket operations, suboptimal scalability in multi-core systems, nonsupport of multiple network interface card (NIC) ports. In this paper, we present FastUDP, a highly efficient and scalable user-level UDP-based network stack optimization in multi-core systems. FastUDP addresses the inefficiencies from the following three novel designs: (1) enabling the exclusive thread model for improving scalability; (2) adopting a poll mode and batched operation for increasing computing resource utilization; (3) constructing a shared hugepage memory pool to eliminate the context switch overhead. Moreover, to support high throughput, FastUDP also proposes a novel work-queue-based approach to allow concurrent packet to transfer over multiple NIC ports. Based on a 40-core machine, the evaluation shows that FastUDP represents a significant improvement in the packet transfer throughput by up to 13× and reduces the packet transfer latency by up to 4.14× compared to the latest Linux (4.4.0) UDP stack. Besides, it ameliorates the performance of realistic application (memcached) by 36 to 67% compared to those on the Linux stack.

Thread Evolution Kit for Optimizing Thread Operations on CE/IoT Devices

Most modern operating systems have adopted the one-to-one thread model to support fast execution of threads in both multi-core and single-core systems. This thread model, which maps the kernel-space and user-space threads in a one-to-one manner, supports quick thread creation and termination in high-performance server environments. However, the performance of time-critical threads is degraded when multiple threads are being run in low-end CE devices with limited system resources. When a CE device runs many threads to support diverse application functionalities, low-level hardware specifications often lead to significant resource contention among the threads trying to obtain system resources. As a result, the operating system encounters challenges, such as excessive thread context switching overhead, execution delay of time-critical threads, and a lack of virtual memory for thread stacks. This paper proposes a state-of-the-art Thread Evolution Kit (TEK) that consists of three primary components: a CPU Mediator, Stack Tuner, and Enhanced Thread Identifier. From the experiment, we can see that the proposed scheme significantly improves user responsiveness (7x faster) under high CPU contention compared to the traditional thread model. Also, TEK solves the segmentation fault problem that frequently occurs when a CE application increases the number of threads during its execution.

Sensory-Specific Satiety, the Variety Effect and Physical Context: Does Change of Context During a Meal Enhance Food Intake?

Food variety has been shown to increase food intake, and sensory-specific satiety (a relative decrease in pleasantness of a food as it is consumed) has been proposed as the mechanism through which variety increases consumption. The aim of this study was to investigate whether variation of eating context can add to experienced meal variety and hence increase consumption even further. A total of 128 participants were assigned to one of four conditions in which they first ate a specific food item (ad libitum) until satiated, after which they consumed a second course ad libitum of either the same or a different food in either the same context or in a different context. We hypothesized that, compared to eating the same food in the same context during the second course, introducing a different food item or changing the context for the second course increases consumption (of the second course), and changing both food and context enhances food intake to a greater degree than only changing the food or changing the context. Results indicated that food variety (introducing a different food) significantly increased consumption in the second course, but that a context switch did not enhance consumption. These results suggest that there is little reason to believe that sensory-specific satiety is context specific.

Cyborgan OS: A Lightweight Real-Time Operating System for Artificial Organ

The software of artificial organ is more and more complex, but it lacks real-time operating system to manage and schedule its resources. In this paper, we propose a lightweight real-time operating system (RTOS) Cyborgan OS based on the SmartOSEK OS. Cyborgan OS optimizes and improves it from the code size, context switch, low power consumption, and partial dynamic update, making it suitable for the artificial organ control system. Finally, we use the heart blood pump model to analyze the task allocation and execution sequence as well as the code size of the whole program. In this application, the maximum space occupied by the code is only 15 kB, which is suitable for most microcontrollers.

Embodied Navigation in Immersive Abstract Data Visualization: Is Overview+Detail or Zooming Better for 3D Scatterplots?

data has no natural scale and so interactive data visualizations must provide techniques to allow the user to choose their viewpoint and scale. Such techniques are well established in desktop visualization tools. The two most common techniques are zoom+pan and overview+detail. However, how best to enable the analyst to navigate and view abstract data at different levels of scale in immersive environments has not previously been studied. We report the findings of the first systematic study of immersive navigation techniques for 3D scatterplots. We tested four conditions that represent our best attempt to adapt standard 2D navigation techniques to data visualization in an immersive environment while still providing standard immersive navigation techniques through physical movement and teleportation. We compared room-sized visualization versus a zooming interface, each with and without an overview. We find significant differences in participants' response times and accuracy for a number of standard visual analysis tasks. Both zoom and overview provide benefits over standard locomotion support alone (i.e., physical movement and pointer teleportation). However, which variation is superior, depends on the task. We obtain a more nuanced understanding of the results by analyzing them in terms of a time-cost model for the different components of navigation: way-finding, travel, number of travel steps, and context switching.

When Storage Response Time Catches Up With Overall Context Switch Overhead, What Is Next?

The virtual memory technique provides a large and cheap memory space by extending the memory space with storage devices. It applies context switch to asynchronously swapping pages between memory and storage devices for hiding the long response time of storage devices when a page fault occurs. However, the overall context switch overhead is high because the context switch itself is a complex function and would further incur TLB shootdown/flush and compulsory CPU cache misses after context switches. On the contrary, as the rapid responsiveness improvement of high-end storage devices, we observe that the response time of high-end storage devices catches up and gradually becomes smaller than the overall context switch overhead. At this turning point, to further enhance the system responsiveness, we advocate adopting synchronous swapping rather than context switch in response to page faults. Meanwhile, we propose a strategy, called shadow huge page management, to further improve the overall system performance by minimizing the overall time overheads caused by page faults and page swappings. Evaluation results show that the proposed system can efficiently reduce the total CPU wasting time.

Synchronization-Aware Task Allocation Techniques for Preemption Control to Reduce Blocking Time in Multiprocessor Real-Time System

Multiprocessor real-time systems receive a great deal of attention. For better utilization of multiprocessors in a real-time context, an optimal approach for scheduling, allocation, and synchronization is required. In this research, a novel heuristic synchronization-aware scheduling has been proposed to reduce the blocking delays in a critical section and also bound to minimize multiple priority inversion. The key idea of this technique is to assign the task set in the same processor that accesses a common shared resource and also access them for the longest period of time; thereby, the global sharing of resource transforms into local sharing. From simulation results, it was concluded that the duration of blocking overheads should be minimized up to 25% to 30% and context switching between processors also reduced up to 10% to 15%. On the basis of result analysis, schedulability, minimization of context switching, and reduced blocking time indicate that the proposed method outperforms the existing methods and does not affect the task completion time.

Reconciling Time Slice Conflicts of Virtual Machines With Dual Time Slice for Clouds

The proliferation of system virtualization poses a new challenge for the coarse-grained time sharing techniques for consolidation, since operating systems are running on virtual CPUs. The current system stack was designed under the assumption that operating systems can seize CPU resources at any moment. However, for the guest operating system on a virtual machine (VM), such assumption cannot be guaranteed, since virtual CPUs of VMs share a limited number of physical cores. Due to the time-sharing of physical cores, the execution of a virtual CPU is not contiguous, with a gap between the virtual and real time spaces. Such a virtual time discontinuity problem leads to significant inefficiency for lock and interrupt handling, which rely on the immediate availability of CPUs whenever the operating system requires computation. To reduce scheduling latencies of virtual CPUs, shortening time slices can be a straightforward strategy, but it may lead to the increased overhead of context switching costs across virtual machines for some workloads. It is challenging to determine a single time slice to satisfy all the VMs. In this article, we propose to have dual time slice to resolve the time slice conflict problem occurred in different types of virtual machines.

Real-Time Scheduling for Periodic Tasks on Uniform Multiprocessors

The problem of scheduling a set of periodic tasks on a uniform multiprocessor system is considered in the present study. Each processor in a uniform multiprocessor system is characterized by its speed or computation capacity, i.e., execution of a job on a processor with speed s for t time units completes s × t units of execution. In the commonly-known partitioned scheduling, each task is assigned to a processor and all of its jobs are required to be executed on that processor. However, partitioning of periodic tasks requires solving the bin-packing problem, which is known to be intractable (NPhard in the strong sense). This paper presents a global scheduling algorithm that transforms a given periodic task system into another using a “task-splitting” (as opposed to the “set-splitting”) technique. Each transformed periodic task system is guaranteed to be scheduled successfully on any uniform multiprocessor using a partitioned scheduling algorithm. The earliest deadline first (EDF) algorithm is chosen for scheduling tasks on each processor. It is proven that the proposed algorithm results in the theoretical-maximum utilization bound on any uniform multiprocessor platform if the platform is “reasonably powerful”. Therefore, the proposed algorithm is optimal in the sense of maximizing achievable utilization. Since the task-splitting technique will incur context switches during runtime, we have also considered the ways of reducing the number of context switches, and suggest a method which can significantly reduce the number of context switches in the schedules generated by the proposed algorithm.

Toward a Microarchitecture for Efficient Execution of Irregular Applications

Given the increasing importance of efficient data-intensive computing, we find that modern processor designs are not well suited to the irregular memory access patterns often found in these algorithms. Applications and algorithms that do not exhibit spatial and temporal memory request locality induce high latency and low memory bandwidth due to the high cache miss rate. In response to the performance penalties inherently present in applications with irregular memory accesses, we introduce a GoblinCore-64 (GC64) architecture and a unique memory hierarchy that are explicitly designed to exploit memory performance from irregular memory access patterns. GC64 provides a pressure-driven hardware-managed concurrency control to minimize pipeline stalls and lower the latency of context switches. A novel memory coalescing model is also introduced to enhance the performance of memory systems via request aggregations. We have evaluated the performance benefits of our approach using a series of 24 benchmarks and the results show nearly 50% memory request reductions and a performance acceleration of up to 14.6×.

A survey on optimizations towards best-effort hardware transactional memory

Transactional memory has been attracting increasing attention in recent years, and it provides optimistic concurrency control schemes for shared-memory parallel programs. The rapid development and wide adoption of transactional memory make this programming paradigm promising for achieving breakthroughs in massively parallel computing. There has been a large number of discussions towards transactional memory systems, which aimed at providing relatively simple and intuitive synchronization construction for shared-memory parallel programs without sacrificing performance. Hardware transactional memory (HTM) has become commercially available in mainstream processors, however, due to several inherent architectural limitations that will abort hardware transactions, such as cache overflows, context switches, hardware as well as software exceptions, etc., nowadays HTM systems come in a best-effort way, which necessitates the adoption of a software fallback path to ensure forward progress. In this paper, we survey state-of-the-art software-side optimizations for best-effort hardware transaction system, as well as several novel performance tuning techniques. Research efforts about joint usage of HTM and non-volatile memory (NVM) are also discussed.

Symbolic Reasoning for Automatic Signal Placement

Explicit signaling between threads is a perennial cause of bugs in concurrent programs. While there are several runtime techniques to automatically notify threads upon the availability of some shared resource, such techniques are not widely-adopted due to their run-time overhead. This paper proposes a new solution based on static analysis for automatically generating a performant explicit-signal program from its corresponding implicit-signal implementation. The key idea is to generate verification conditions that allow us to minimize the number of required signals and unnecessary context switches, while guaranteeing semantic equivalence between the source and target programs. We have implemented our method in a tool called Expresso and evaluate it on challenging benchmarks from prior papers and open-source software. Expresso-generated code significantly outperforms past automatic signaling mechanisms (avg. 1.56x speedup) and closely matches the performance of hand-optimized explicit-signal code.

Concepts of variation control systems

Version control systems are an integral part of today’s software engineering. They facilitate the collaborative management of revisions (sequential versions) and variants (concurrent versions) of software systems under development. Typical version control systems maintain revisions of files and variants of whole software systems. Variants are supported via branching or forking mechanisms that conceptually clone whole systems in a coarse-grained way. Unfortunately, such cloning leads to high maintenance efforts. To avoid these disadvantages and support fine-grained variation, developers need to employ custom configuration mechanisms, which leads to a misappropriation of tools and undesired context switches. Addressing this trade-off, a number of variation control systems has been conceived, providing a richer set of capabilities for handling variants. Variation control systems decompose a software system into finer-grained variable entities and offer high-level metaphors to automatically manage this variability. In this paper, we classify and compare variation control systems and illustrate their core concepts and characteristics. All investigated variation control systems offer an iterative (checkout–modify–commit) workflow, but there are essential differences affecting developers. We highlight challenges and discuss research perspectives for developing the next generation of version and variation control systems.

Analyzing the Performance Trade-Off in Implementing User-Level Threads

User-level threads have been widely adopted as a means of achieving lightweight concurrent execution without the costs of OS-level threads. Nevertheless, the costs of managing user-level threads represent a performance barrier that dictates how fine grained the concurrency exposed by an application can be without incurring significant overheads; this in turn may translate into insufficient parallelism to exploit highly parallel systems. This article is a deep dive into the fundamental costs in implementing user-level threads. We first identify that one of the highest sources of fork-join overheads stems from deviations , events that incur context switching during the execution of a thread and disrupt a run-to-completion execution. We then conduct an in-depth investigation of a wide spectrum of methods with respect to how they handle deviations while covering both parent- and child-first scheduling policies. Our methodology involves a comprehensive instruction- and cache-level analysis of all methods on several modern CPU architectures. The primary finding of our evaluation is that dynamic promotion methods that assume the absence of deviation and dynamically provide context-switching support offer the best trade-off between performance and capability when the likelihood of deviation is low.

Alibaba hologres

In existing big data stacks, the processes of analytical processing and knowledge serving are usually separated in different systems. In Alibaba, we observed a new trend where these two processes are fused: knowledge serving incurs generation of new data, and these data are fed into the process of analytical processing which further fine tunes the knowledge base used in the serving process. Splitting this fused processing paradigm into separate systems incurs overhead such as extra data duplication, discrepant application development and expensive system maintenance. In this work, we propose Hologres, which is a cloud native service for hybrid serving and analytical processing (HSAP). Hologres decouples the computation and storage layers, allowing flexible scaling in each layer. Tables are partitioned into self-managed shards. Each shard processes its read and write requests concurrently independent of each other. Hologres leverages hybrid row/column storage to optimize operations such as point lookup, column scan and data ingestion used in HSAP. We propose Execution Context as a resource abstraction between system threads and user tasks. Execution contexts can be cooperatively scheduled with little context switching overhead. Queries are parallelized and mapped to execution contexts for concurrent execution. The scheduling framework enforces resource isolation among different queries and supports customizable schedule policy. We conducted experiments comparing Hologres with existing systems specifically designed for analytical processing and serving workloads. The results show that Hologres consistently outperforms other systems in both system throughput and end-to-end query latency.

Dynamic Round Robin CPU Scheduling Algorithm Based on K-Means Clustering Technique

Minimizing time cost in time-shared operating system is the main aim of the researchers interested in CPU scheduling. CPU scheduling is the basic job within any operating system. Scheduling criteria (e.g., waiting time, turnaround time and number of context switches (NCS)) are used to compare CPU scheduling algorithms. Round robin (RR) is the most common preemptive scheduling policy used in time-shared operating systems. In this paper, a modified version of the RR algorithm is introduced to combine the advantageous of favor short process and low scheduling overhead of RR for the sake of minimizing average waiting time, turnaround time and NCS. The proposed work starts by clustering the processes into clusters where each cluster contains processes that are similar in attributes (e.g., CPU service period, weights and number of allocations to CPU). Every process in a cluster is assigned the same time slice depending on the weight of its cluster and its CPU service period. The authors performed comparative study of the proposed approach and popular scheduling algorithms on nine groups of processes vary in their attributes. The evaluation was measured in terms of waiting time, turnaround time, and NCS. The experiments showed that the proposed approach gives better results.

Aging-Aware Context Switching in Multicore Processors Based on Workload Classification

As transistor dimensions continue to shrink, long-term reliability threats, such as Negative Bias Temperature Instability, affect multicore processors lifespan. This letter proposes a load balancing technique, based on the rate of integer and floating-point instructions per workloads. This technique classifies workloads into integer-majority and floating-point-majority classes and migrates workloads among cores in order to relax the stressed execution units. The context switching feature of operating system is employed to reduce implementation and performance overheads of the proposed technique. According to the simulations, the proposed technique reduces the aging rate of a multicore processor by about 35 percent in 10 years of system operation. This is achieved by only 7 percent performance penalty.