Spatial Multitasking Research Articles

Investigating traffic and controller factors in spatial multitasking: The context of air traffic conflict resolution

Air traffic control (ATC) operations including conflict detection and resolution mainly consist of spatial and multi tasks. The analysis of gender differences in performing spatial and multi tasks in the context of air traffic control remain absent. This study aimed to investigate the influence of traffic factor (i.e., conflict geometry) and controller factor (i.e., gender differences) on the objective and physiological measures of the controllers. Twenty-two college students consisted of eleven male and females, respectively, performed conflict resolution tasks given six different conflict geometries (i.e., crossing, converging, and overtaking conflicts for both level and non-level courses). The results indicated that the effects of gender were absent on the objective measures (i.e., controllers’ performance, workload, and situation awareness) but present on the physiological measure. Gender differences affected beta activation on the left prefrontal ( p = 0.027) and right parietal lobes ( p = 0.008) where male controllers exhibit higher activation as compared to the female counterparts. These findings offer several implications for conflict resolution in ATC including function allocation and training as well as technological intervention.

NURA

Multi-application execution in Graphics Processing Units (GPUs), a promising way to utilize GPU resources, is still challenging. Some pieces of prior work (e.g. spatial multitasking) have limited opportunity to improve resource utilization, while others, e.g. simultaneous multi-kernel, provide fine-grained resource sharing at the price of unfair execution. This paper proposes a new multi-application paradigm for GPUs, called NURA, that provides high potential to improve resource utilization and ensure fairness and Quality-of-Service(QoS). The key idea is that each streaming multiprocessor (SM) executes the Cooperative Thread Arrays (CTAs) that belong to only one application (similar to spatial multi-tasking) and shares its unused resources with the SMs running other applications demanding more resources. NURA handles resource sharing process mainly using a software approach to provide simplicity, low hardware overhead, and flexibility.We also perform some hardware modifications as an architectural support for our software-based proposal. Our conservative analysis reveals that the hardware area overhead of our proposal is less than 1.07% with respect to the whole GPU die. Our experimental results over various mixes of GPU workloads show that NURA improves throughput by 26% compared to the state-of-the-art spatial multi-tasking, on average, while meeting QoS targets. In terms of fairness, NURA has almost similar results to spatial multitasking, while it outperforms simultaneous multi-kernel by 76%, on average.

A Survey of GPU Multitasking Methods Supported by Hardware Architecture

The ability to support multitasking becomes more and more important in the development of graphic processing unit (GPU). GPU multitasking methods are classified into three types: temporal multitasking, spatial multitasking, and simultaneous multitasking (SMK). This article first introduces the features of some commercial GPU architectures to support multitasking and the common metrics used for evaluating the performance of GPU multitasking methods, and then reviews the GPU multitasking methods supported by hardware architecture (i.e., hardware GPU multitasking methods). The main problems of each type of hardware GPU multitasking methods to be solved are illustrated. Meanwhile, the key idea of each previous hardware GPU multitasking method is introduced. In addition, the characteristics of hardware GPU multitasking methods belonging to the same type are compared. This article also gives some valuable suggestions for the future research. An enhanced GPU simulator is needed to bridge the gap between academia and industry. In addition, it is promising to expand the research space with machine learning technologies, advanced GPU architectural innovations, 3D stacked memory, etc. Because most previous GPU multitasking methods are based on NVIDIA GPUs, this article focuses on NVIDIA GPU architecture, and uses NVIDIA's terminology. To our knowledge, this article is the first survey about hardware GPU multitasking methods. We believe that our survey can help the readers gain insights into the research field of hardware GPU multitasking methods.

NURA

Multi-application execution in Graphics Processing Units (GPUs), a promising way to utilize GPU resources, is still challenging. Some pieces of prior work (e.g., spatial multitasking) have limited opportunity to improve resource utilization, while other works, e.g., simultaneous multi-kernel, provide fine-grained resource sharing at the price of unfair execution. This paper proposes a new multi-application paradigm for GPUs, called NURA, that provides high potential to improve resource utilization and ensures fairness and Quality-of-Service (QoS). The key idea is that each streaming multiprocessor (SM) executes Cooperative Thread Arrays (CTAs) belong to only one application (similar to the spatial multi-tasking) and shares its unused resources with the SMs running other applications demanding more resources. NURA handles resource sharing process mainly using a software approach to provide simplicity, low hardware cost, and flexibility. We also perform some hardware modifications as an architectural support for our software-based proposal. We conservatively analyze the hardware cost of our proposal, and observe less than 1.07% area overhead with respect to the whole GPU die. Our experimental results over various mixes of GPU workloads show that NURA improves GPU system throughput by 26% compared to state-of-the-art spatial multi-tasking, on average, while meeting the QoS target. In terms of fairness, NURA has almost similar results to spatial multitasking, while it outperforms simultaneous multi-kernel by an average of 76%.

Toward QoS-Awareness and Improved Utilization of Spatial Multitasking GPUs

Datacenters use GPUs to provide the significant computing throughput required by emerging user-facing services. The diurnal user access pattern of user-facing services provides a strong incentive to co-located applications for better GPU utilization, and prior work has focused on enabling co-location on multicore processors and traditional non-preemptive GPUs. However, current GPUs are evolving towards spatial multitasking and introduce a new set of challenges to eliminate QoS violations. To address this open problem, we explore the underlying causes of QoS violation on spatial multitasking GPUs. In response to these causes, we propose C-Laius, a runtime system that carefully allocates the computation resource to co-located applications for maximizing the throughput of batch applications while guaranteeing the required QoS of user-facing services. C-Laius not only allows co-locating one user-facing application with multiple batch applications, but also supports the co-location of multiple user-facing applications with batch applications. In the case of a single co-located user-facing application, our evaluation on an Nvidia RTX 2080Ti GPU shows that C-Laius improves the utilization of spatial multitasking GPUs by 20.8 percent, while achieving the 99%-ile latency target for user-facing services. As to the case of multiple co-located user-facing applications, C-Laius ensures no violation of QoS while improving the accelerator utilization by 35.9 percent on average.

Fair and cache blocking aware warp scheduling for concurrent kernel execution on GPU

With Graphic Processing Units (GPUs) being widely adopted in data centers to provide computing power, efficient support for GPU multitasking attracts significant attention. The prior GPU multitasking works include spatial multitasking and simultaneous multitasking (SMK). Spatial multitasking allocates GPU resources at the streaming multiprocessor (SM) granularity which is coarse-grained, and SMK runs concurrent kernels on the same SM, therefore is fine-grained. SMK is beneficial to improve GPU resource utilization especially when concurrent kernels have complementary characteristics. However, the main challenge for SMK is the interference among multiple kernels especially the contention on data cache. In this paper, we propose a fair and cache blocking aware warp scheduling (FCBWS) approach to ameliorate the contention on data cache and improve SMK on GPUs. In FCBWS, equal opportunity of issuing instructions is provided to each kernel, and memory pipeline stalls are tried to be avoided by predicting cache blocking. Kernels are extracted from various applications to construct concurrent kernel execution benchmarks. The simulation experiment results show that FCBWS outperforms previous multitasking methods; even compared to the state-of-the-art SMK method, FCBWS can improve system throughput (STP) by 10% on average and reduce average normalized turnaround time (ANTT) by 41% on average.

Predicting and reining in application-level slowdown on spatial multitasking GPUs

Predicting performance degradation of a GPU application at co-location on a spatial multitasking GPU without prior application knowledge is essential in public Clouds. Prior work mainly targets CPU co-location, and is inaccurate and/or inefficient for predicting performance of applications at co-location on spatial multitasking GPUs. Our investigation shows that hardware event statistics caused by co-located applications strongly correlate with their slowdowns. Based on this observation, we present Themis with a kernel slowdown model (Themis-KSM), which performs precise and efficient online application slowdown prediction without prior application knowledge. The kernel slowdown model is trained offline. When new applications co-run, Themis-KSM collects event statistics and predicts their slowdowns simultaneously. In addition, we also propose a two-stage slowdown prediction mechanism (Themis-TSP) for real-system GPUs without any hardware modification. Our evaluation shows that Themis has negligible runtime overhead, and both Themis-KSM and Themis-TSP can precisely predict application-level slowdown with prediction error smaller than 9.5% and 12.8%, respectively. Based on Themis, we also implement an SM allocation engine to rein in application slowdown at co-location. Case studies show that the engine successfully enforces fair sharing and QoS.

Improving GPU Multitasking Efficiency Using Dynamic Resource Sharing

As GPUs have become essential components for embedded computing systems, a shared GPU with multiple CPU cores needs to efficiently support concurrent execution of multiple different applications. Spatial multitasking, which assigns a different amount of streaming multiprocessors (SMs) to multiple applications, is one of the most common solutions for this. However, this is not a panacea for maximizing total resource utilization. It is because an SM consists of many different sub-resources such as caches, execution units and scheduling units, and the requirements of the sub-resources per kernel are not well matched to their fixed sizes inside an SM. To solve the resource requirement mismatch problem, this paper proposes a GPU Weaver , a dynamic sub-resource management system of multitasking GPUs. GPU Weaver can maximize sub-resource utilization through a shared resource controller (SRC) that is added between neighboring SMs. The SRC dynamically identifies idle sub-resources of an SM and allows them to be used by the neighboring SM when possible. Experiments show that the combination of multiple sub-resource borrowing techniques enhances the total throughput by up to 26 and 9.5 percent on average over the baseline spatial multitasking GPU.

KSM: Online Application-Level Performance Slowdown Prediction for Spatial Multitasking GPGPU

Colocating multiple applications on the same spatial multitasking GPGPU improves the system-wide throughput. However, the colocated applications are slowed down differently due to the contention on streaming multiprocessors (SMs), L2 cache and global memory bandwidth. The ability to precisely predict application slowdowns online is useful in many scenarios, e.g., ensuring fair pricing in multi-tenant Cloud systems. Prior work on predicting application slowdown is either inaccurate, due to the ignoring of contention on SMs, or inefficient, due to the expensive sequential profiling of concurrent applications via runtime environment switching. To solve the above problem, we propose KSM that enables precise and efficient application-level slowdown prediction without priori application knowledge. KSM is proposed based on the observation that hardware event statistics caused by the colocated applications are strongly correlated with their slowdowns. In more detail, KSM builds a slowdown model based on the hardware event statistics using machine learning techniques offline. At runtime, KSM collects the hardware event statistics, and predicts the slowdowns of all the colocated applications based on the model. Our experimental results show that KSM has negligible runtime overhead and precisely predicts the application-level slowdowns with the prediction error smaller than 9.7 percent.

Dynamic Resource Management for Efficient Utilization of Multitasking GPUs

As graphics processing units (GPUs) are broadly adopted, running multiple applications on a GPU at the same time is beginning to attract wide attention. Recent proposals on multitasking GPUs have focused on either spatial multitasking, which partitions GPU resource at a streaming multiprocessor (SM) granularity, or simultaneous multikernel (SMK), which runs multiple kernels on the same SM. However, multitasking performance varies heavily depending on the resource partitions within each scheme, and the application mixes. In this paper, we propose GPU Maestro that performs dynamic resource management for efficient utilization of multitasking GPUs. GPU Maestro can discover the best performing GPU resource partition exploiting both spatial multitasking and SMK. Furthermore, dynamism within a kernel and interference between the kernels are automatically considered because GPU Maestro finds the best performing partition through direct measurements. Evaluations show that GPU Maestro can improve average system throughput by 20.2% and 13.9% over the baseline spatial multitasking and SMK, respectively.

Dynamic Resource Management for Efficient Utilization of Multitasking GPUs

As graphics processing units (GPUs) are broadly adopted, running multiple applications on a GPU at the same time is beginning to attract wide attention. Recent proposals on multitasking GPUs have focused on either spatial multitasking, which partitions GPU resource at a streaming multiprocessor (SM) granularity, or simultaneous multikernel (SMK), which runs multiple kernels on the same SM. However, multitasking performance varies heavily depending on the resource partitions within each scheme, and the application mixes. In this paper, we propose GPU Maestro that performs dynamic resource management for efficient utilization of multitasking GPUs. GPU Maestro can discover the best performing GPU resource partition exploiting both spatial multitasking and SMK. Furthermore, dynamism within a kernel and interference between the kernels are automatically considered because GPU Maestro finds the best performing partition through direct measurements. Evaluations show that GPU Maestro can improve average system throughput by 20.2% and 13.9% over the baseline spatial multitasking and SMK, respectively.

Warped-slicer

As technology scales, GPUs are forecasted to incorporate an ever-increasing amount of computing resources to support thread-level parallelism. But even with the best effort, exposing massive thread-level parallelism from a single GPU kernel, particularly from general purpose applications, is going to be a difficult challenge. In some cases, even if there is sufficient thread-level parallelism in a kernel, there may not be enough available memory bandwidth to support such massive concurrent thread execution. Hence, GPU resources may be underutilized as more general purpose applications are ported to execute on GPUs. In this paper, we explore multiprogramming GPUs as a way to resolve the resource underutilization issue. There is a growing hardware support for multiprogramming on GPUs. Hyper-Q has been introduced in the Kepler architecture which enables multiple kernels to be invoked via tens of hardware queue streams. Spatial multitasking has been proposed to partition GPU resources across multiple kernels. But the partitioning is done at the coarse granularity of streaming multiprocessors (SMs) where each kernel is assigned to a subset of SMs. In this paper, we advocate for partitioning a single SM across multiple kernels, which we term as intra-SM slicing. We explore various intra-SM slicing strategies that slice resources within each SM to concurrently run multiple kernels on the SM. Our results show that there is not one intra-SM slicing strategy that derives the best performance for all application pairs. We propose Warped-Slicer , a dynamic intra-SM slicing strategy that uses an analytical method for calculating the SM resource partitioning across different kernels that maximizes performance. The model relies on a set of short online profile runs to determine how each kernel's performance varies as more thread blocks from each kernel are assigned to an SM. The model takes into account the interference effect of shared resource usage across multiple kernels. The model is also computationally efficient and can determine the resource partitioning quickly to enable dynamic decision making as new kernels enter the system. We demonstrate that the proposed Warped-Slicer approach improves performance by 23% over the baseline multiprogramming approach with minimal hardware overhead.

Efficient GPU Spatial-Temporal Multitasking

Heterogeneous computing nodes are now pervasive throughout computing, and GPUs have emerged as a leading computing device for application acceleration. GPUs have tremendous computing potential for data-parallel applications, and the emergence of GPUs has led to proliferation of GPU-accelerated applications. This proliferation has also led to systems in which many applications are competing for access to GPU resources, and efficient utilization of the GPU resources is critical to system performance. Prior techniques of temporal multitasking can be employed with GPU resources as well, but not all GPU kernels make full use of the GPU resources. There is, therefore, an unmet need for spatial multitasking in GPUs. Resources used inefficiently by one kernel can be instead assigned to another kernel that can more effectively use the resources. In this paper we propose a software-hardware solution for efficient spatial-temporal multitasking and a software based emulation framework for our system. We pair an efficient heuristic in software with hardware leaky-bucket based thread-block interleaving to implement spatial-temporal multitasking. We demonstrate our techniques on various GPU architecture using nine representative benchmarks from CUDA SDK. Our experiments on Fermi GTX480 demonstrate performance improvement by up to 46% (average 26%) over sequential GPU task execution and 37% (average 18%) over default concurrent multitasking. Compared with the state-of-the-art Kepler K20 using Hyper-Q technology, our technique achieves up to 40% (average 17%) performance improvement over default concurrent multitasking.

Postural prioritization is differentially altered in healthy older compared to younger adults during visual and auditory coded spatial multitasking

Many daily activities require appropriate allocation of attention between postural and cognitive tasks (i.e. dual-tasking) to be carried out effectively. Processing multiple streams of spatial information is important for everyday tasks such as road crossing. Fifteen community-dwelling healthy older (mean age=78.3, male=1) and twenty younger adults (mean age=25.3, male=6) completed a novel bimodal spatial multi-task test providing contextually similar spatial information via separate sensory modalities to investigate effects on postural prioritization. Two tasks, a temporally random visually coded spatial step navigation task (VS) and a regular auditory-coded spatial congruency task (AS) were performed independently (single task) and in combination (multi-task). Response time, accuracy and dual-task costs (% change in multi-task condition) were determined. Results showed a significant 3-way interaction between task type (VS vs. AS), complexity (single vs. multi) and age group for both response time (p≤0.01) and response accuracy (p≤0.05) with older adults performing significantly worse than younger adults. Dual-task costs were significantly greater for older compared to younger adults in the VS step task for both response time (p≤0.01) and accuracy (p≤0.05) indicating prioritization of the AS over the VS stepping task in older adults. Younger adults display greater AS task response time dual task costs compared to older adults (p≤0.05) indicating VS task prioritization in agreement with the posture first strategy. Findings suggest that novel dual modality spatial testing may lead to adoption of postural strategies that deviate from posture first, particularly in older people. Adoption of previously unreported postural prioritization strategies may influence balance control in older people.