Cloud Gaming Research Articles

A Network Analysis on Cloud Gaming: Stadia, GeForce Now and PSNow

Cloud gaming is a class of services that promises to revolutionize the videogame market. It allows the user to play a videogame with essential equipment while using a remote server for the actual execution. The multimedia content is streamed through the network from the server to the user. Hence, this service requires low latency and a large bandwidth to work properly with low response time and high-definition video. Three of the leading tech companies (Google, Sony, and NVIDIA) entered this market with their products, and others, like Microsoft and Amazon, are also launching their platforms. However, these companies have released little information about their cloud gaming operation and how they utilize the network. In this work, we study cloud gaming services from the network point of view. We collect more than 200 packet traces under different application settings and network conditions from a broadband network to poor mobile network conditions, for 3 cloud gaming services, namely Stadia from Google, GeForce Now from NVIDIA and PS Now from Sony. We analyze the employed protocols and the workload that they impose on the network. We find that GeForce Now and Stadia use the RTP protocol to stream the multimedia content, with the latter relying on the standard WebRTC APIs. Depending on the network and video quality, they result in bandwidth-hungry services consuming up to 45 Mbit/s. PS Now instead uses only undocumented protocols and never exceeds 13 Mbit/s. 4G mobile networks can often sustain these loads, while traditional 3G connections struggle. The systems quickly react to deteriorated network conditions, and packet losses up to 5% do not cause a reduction in resolution.

Increasing Player Performance and Game Experience in High Latency Systems

Cloud gaming services and remote play offer a wide range of advantages but can inherent a considerable delay between input and action also known as latency. Previous work indicates that deep learning algorithms such as artificial neural networks (ANN) are able to compensate for latency. As high latency in video games significantly reduces player performance and game experience, this work investigates if latency can be compensated using ANNs within a live first-person action game. We developed a 3D video game and coupled it with the prediction of an ANN. We trained our network on data of 24 participants who played the game in a first study. We evaluated our system in a second user study with 96 participants. To simulate latency in cloud game streaming services, we added 180 ms latency to the game by buffering user inputs. In the study we predicted latency values of 60 ms, 120 ms and 180 ms. Our results show that players achieve significantly higher scores, substantially more hits per shot and associate the game significantly stronger with a positive affect when supported by our ANN. This work illustrates that high latency systems, such as game streaming services, benefit from utilizing a predictive system.

Energy and Quality Aware Multi-Objective Resource Allocation Algorithm in Cloud

Cloud gaming has become the new service provisioning prototype that hosts the video games in the cloud and broadcasts the interactive game streaming to the players through the Internet. Here, the cloud must use massive resources for video representation and its streaming when several simultaneous players reach a particular point. Alternatively, various players may have separate necessities on Quality-of Experience, like low delay, high-video quality, etc. The challenging task is providing better service by the fixed cloud resource. Hence, there is a necessity for an energy-aware multi-resource allocation in the cloud. This paper devises a Fractional Rider-Harmony search algorithm (Fractional Rider-HSA) for resource allocation in the cloud. The Fractional Rider-HSA combines fractional calculus, Rider Optimization algorithm (ROA), and HSA. Moreover, the fitness function, like mean opinion score (MOS), gaming experience loss, fairness, energy consumption, and network parameters, is considered to determine the optimal resource allocation. The proposed model produces the maximal MOS of 0.8961, maximal gaming experience loss (QE) of 0.998, maximal fairness of 0.9991, the minimum energy consumption of 0.3109, and minimal delay 0.2266, respectively.

Enabling and Scaling of URLLC Verticals on 5G vRAN Running on COTS Hardware

This article discusses an Ultra-Reliable Low-La-tency Communication (URLLC) implementation on a 5G New Radio (NR) virtual radio access Network (vRAN). The vRAN application runs on a commercial off-the-shelf (COTS) test platform, on which 3GPP 5G NR RAN Layer-1 (L1) and Layer-2 (L2) containerized network functions (CNFs) are implemented. To demonstrate capability and scalability, the CNFs are dimensioned to support up to three 100 MHz cells. URLLC and enhanced mobile broadband (eMBB) slices are implemented to partition L1 processing for mixed use cases. This article illustrates an advantage of virtualization in managing L1 processing demand by L1 slice-scaling. It also describes a methodology for optimal cell capacity dimensioning based on connected URLLC users' service level agreements (SLAs) in terms of latency, reliability, and throughput. Using analysis of the test results, the article discusses maintaining a critical time budget of downlink (DL) and uplink (UL) L1 processing pipelines and points out techniques to increase L1 processing capacity to achieve URLLC SLA while serving eMBB users. The article concludes that a 5G NR vRAN application running on COTS hardware enables public/private URLLC verticals including mobile edge computing, cloud gaming, AR/VR, industrial automation, and others.

A hybrid graphics/video rate control method based on graphical assets for cloud gaming

A hybrid graphics/video rate control method based on graphical assets for cloud gaming

Subjective Quality Assessment for Cloud Gaming

Using subjective testing, we study the effect of the network parameters, delay and packet loss ratio, on the QoE of cloud gaming. We studied three different games, selected based on genre, popularity, content complexity and pace, and tested them in a controlled network environment, using a novel emulator to create realistic lognormal delay distributions instead of relying on a static mean delay, as used previously; we also used Parsec as a good representative of the state of the art. We captured user ratings on an ordinal Absolute Category Rating scale for three quality dimensions: Video QoE, Game-Playability QoE, and Overall QoE. We show that Mean Opinion Scores (MOS) for the game with the highest levels of content complexity and pace are most severely affected by network impairments. We also show that the QoE of interactive cloud applications rely more on the game playability than the video quality of the game. Unlike earlier studies, the differences in MOS are validated using the distributions of the underlying dimensions. A Wilcoxon Signed-Rank test showed that the distributions of Video QoE and Game Playability QoE are not significantly different.

QoS and QoE aware multi objective resource allocation algorithm for cloud gaming

Cloud gaming is an innovative model that congregates video games. The user may have different Quality-of-Experience (QoE), which is a term used to measure a user’s level of satisfaction and enjoyment for a particular service. To guarantee general satisfaction for all users with limited cloud resources, it becomes a major issue in the cloud. This paper leverages a game theory in the cloud gaming model with resource optimization to discover optimal solutions to resolve resource allocation. The Rider-based harmony search algorithm (Rider-based HSA), which is the combination of Rider optimization algorithm (ROA) and Harmony search algorithm (HSA), is proposed for resource allocation to improve the cloud computing system’s efficiency. The fitness function is newly devised considering certain QoE parameters, which involves fairness index, Quantified experience of players (QE), and Mean Opinion Score (MOS). The proposed Rider-based HSA showed better performance compared to Potential game-based optimization algorithm, Proactive resource allocation algorithm, QoE-aware resource allocation algorithm, Distributed algorithm, and Yusen Li et al., with maximal fairness of 0.999, maximal MOS of 0.873, and maximal QE of 1.

An Elastic System Architecture for Edge Based Low Latency Interactive Video Applications

5G and edge computing have brought great changes to video industry. Interactive video is becoming an emerging application form of multimedia service, which provides attraction beyond typical scenarios like cloud gaming and remote virtual reality (VR), and puts forward great challenges in resource capacity, response latency, and function flexibility to its service system. In this paper, we propose an elastic system architecture with low latency features to accommodate generic interactive video applications on near user edges. To increase system flexibility, we firstly design a dynamic Directed Acyclic Graph (dDAG) model for efficient task representation. Secondly, based on the model, we present the elastic architecture together with its scalable workflow pipeline. Thirdly, we propose a set of novel latency measurement metrics to analyze and optimize the performance of an interactive video system. Based on the proposed approaches, we disassemble a real world free-viewpoint synthesis application and benchmark its performance with the metrics. Extensive experimental results show the flexibility of our system to handle the stochastic human interactions during a video service session, with less than 5 ms additional scheduling latency introduced. End to end latency is kept within 43 ms for complex functions, and 28 ms for simpler scenarios, which satisfies the restrictions of most interactive video applications provided by an edge. Client of the architecture serves as a pure video player, which is also friendly to power limited terminals such as 5G phones. Efficiency and stability analyses of the system show superiorities over existing work, and also reveal potential optimization directions for future research.

Server Allocation for Massively Multiplayer Online Cloud Games Using Evolutionary Optimization

In recent years, Massively Multiplayer Online Games (MMOGs) are becoming popular, partially due to their sophisticated graphics and broad virtual world, and cloud gaming is demanded more than ever especially when entertaining with light and portable devices. This article considers the problem of server allocation for running MMOG on cloud, aiming to reduce the cost on cloud gaming service and meanwhile enhance the quality of service. The problem is formulated into minimizing an objective function involving the cost of server rental, the cost of data transfer and the network latency during the gaming time. A genetic algorithm is developed to solve the minimization problem for processing simultaneous server allocation for the players who log into the system at the same time while many existing players are playing the same game. Extensive experiments based on the player behavior in “World of Warcraft” are conducted to evaluate the proposed method and compare with the state-of-the-art as well. The experimental results show that the method gives a lower cost and a shorter network latency in most of the time.

NetworkAPI: An In-Band Signalling Application-Aware Traffic Engineering Using SRv6 and IP Anycast

Application-aware Traffic Engineering (TE) plays a crucial role in ensuring quality of services (QoS) for recently emerging applications such as AR, VR, cloud gaming, and connected vehicles. While a deterministic application-aware TE is required for these mission-critical applications, a negotiation procedure between applications and network operators needs to undergo major simplification to fulfill the scalability of the application based on emerging microservices and container-based architecture. In this paper, we propose a NetworkAPI framework which allows an application to indicate a desired TE behavior inside IP packets by leveraging Segment Routing over IPv6 (SRv6). In the NetworkAPI framework, the TE behavior provided by the network operator is expressed as an SRv6 Segment Identifier (SID) in the form of a 128-bit IPv6 address. Because the IPv6 address of an SRv6 SID is distributed using IP anycast, the application can utilize the unchanged SRv6 SID regardless of the application's location, as if the application controls an API on the transport network. We implement a prototype of the NetworkAPI framework on a Linux kernel. On the prototype implementation, a basic packet forwarding performance is evaluated to demonstrate the feasibility of our framework.

Progressive Traffic-Oriented Resource Management for Reducing Network Congestion in Edge Computing.

Edge computing can deliver network services with low latency and real-time processing by providing cloud services at the network edge. Edge computing has a number of advantages such as low latency, locality, and network traffic distribution, but the associated resource management has become a significant challenge because of its inherent hierarchical, distributed, and heterogeneous nature. Various cloud-based network services such as crowd sensing, hierarchical deep learning systems, and cloud gaming each have their own traffic patterns and computing requirements. To provide a satisfactory user experience for these services, resource management that comprehensively considers service diversity, client usage patterns, and network performance indicators is required. In this study, an algorithm that simultaneously considers computing resources and network traffic load when deploying servers that provide edge services is proposed. The proposed algorithm generates candidate deployments based on factors that affect traffic load, such as the number of servers, server location, and client mapping according to service characteristics and usage. A final deployment plan is then established using a partial vector bin packing scheme that considers both the generated traffic and computing resources in the network. The proposed algorithm is evaluated using several simulations that consider actual network service and device characteristics.

Reinforcement Learning-Based Resource Partitioning for Improving Responsiveness in Cloud Gaming

Cloud gaming has been very popular in recent years, but issues relating to maintaining low interaction delay to guarantee satisfactory user experience are still prevalent. We observe that the server-side processing delay in cloud gaming system could be heavily influenced by how the resources are partitioned among processes. However, finding the optimal partitioning policy that minimizes the response delay faces several critical challenges. In this paper, we propose an online resource partitioning framework for reducing response delay in cloud gaming, which has several promising properties. First, we divide the processes into disjoint groups and partition resources among process groups, which greatly simplifies the resource partitioning problem while ensuring high partitioning effectiveness. Second, to tackle dynamic workload changes, we classify game workloads into several clusters and maintain separate process grouping plan for each cluster. Third, we leverage reinforcement learning to adaptively choose the best actions for minimizing response delay in real time. We evaluate the proposed framework in a real cloud gaming environment using several real games. The experimental results show that our approach can reduce the response delay by 22\% to 41\% compared to a system without resource partitioning, and outperforms other resource partitioning policies significantly.

Measuring Key Quality Indicators in Cloud Gaming: Framework and Assessment Over Wireless Networks.

Cloud Gaming is a cutting-edge paradigm in the video game provision where the graphics rendering and logic are computed in the cloud. This allows a user’s thin client systems with much more limited capabilities to offer a comparable experience with traditional local and online gaming but using reduced hardware requirements. In contrast, this approach stresses the communication networks between the client and the cloud. In this context, it is necessary to know how to configure the network in order to provide service with the best quality. To that end, the present work defines a novel framework for Cloud Gaming performance evaluation. This system is implemented in a real testbed and evaluates the Cloud Gaming approach for different transport networks (Ethernet, WiFi, and LTE (Long Term Evolution)) and scenarios, automating the acquisition of the gaming metrics. From this, the impact on the overall gaming experience is analyzed identifying the main parameters involved in its performance. Hence, the future lines for Cloud Gaming QoE-based (Quality of Experience) optimization are established, this way being of configuration, a trendy paradigm in the new-generation networks, such as 4G and 5G (Fourth and Fifth Generation of Mobile Networks).

GRemote: Cloud rendering on GPU resource pool based on API-forwarding

Traditional GPU resource allocation approaches, widely adopted in today’s data centers as the rise of cloud gaming, only focus on server-side functions while ignoring the client-side, which wastes hardware resources. To solve this problem, cloud-edge integrated architectures are put forward, leaving some workloads to the client-side. However, many cloud-edge integrated frameworks suffer from one big issue: shared-resource interference, stemming from two reasons: (a) GPU resource racing caused by resources overuse for single client, and (b) CPU resource racing caused by resources shortage among clients. This paper presents gRemote, an open-source cloud-rendering system that can address these issues on a GPU resource pool. To mitigate the CPU resource shortage, gRemote improves CPU configuration by expanding CPU resources from server-side to both server- and client-side. To maintain the reasonable GPU usage for individual tasks, gRemote innovates a new resource-sharing mechanism called GPU throttle. Furthermore, remote API-Forwarding brings another type of network bandwidth consumption: command streaming. To optimize the network bandwidth for GPU command transmission, gRemote proposes two methods, command characteristics-oriented (CCO) compression algorithm and command-transmitted methodology. gRemote supports 1,228 OpenGL commands and provides cloud rendering for more than 40 clients with negligible shared-resource interference. With CCO algorithm and command-transmitted methodology, the network bandwidth is saved by more than 90%.

SVM based approach for complexity control of HEVC intra coding

The High Efficiency Video Coding (HEVC) is adopted by various video applications in recent years. Because of its high computational demand, controlling the complexity of HEVC is of paramount importance to appeal to the varying requirements in many applications, including power-constrained video coding, video streaming, and cloud gaming. Most of the existing complexity control methods are only capable of considering a subset of the decision space, which leads to low coding efficiency. While the efficiency of machine learning methods such as Support Vector Machines (SVM) can be employed for higher precision decision making, the current SVM-based techniques for HEVC provide a fixed decision boundary which results in different coding complexities for different video content. Although this might be suitable for complexity reduction, it is not acceptable for complexity control. This paper proposes an adjustable classification approach for Coding Unit (CU) partitioning, which addresses the mentioned problems of complexity control. Firstly, a novel set of features for fast CU partitioning is designed using image processing techniques. Then, a flexible classification method based on SVM is proposed to model the CU partitioning problem. This approach allows adjusting the performance-complexity trade-off, even after the training phase. Using this model, and a novel adaptive thresholding technique, an algorithm is presented to deliver video encoding within the target coding complexity, while maximizing the coding efficiency. Experimental results justify the superiority of this method over the state-of-the-art methods, with target complexities ranging from 20% to 100%.

Towards Minimizing Resource Usage With QoS Guarantee in Cloud Gaming

Cloud gaming has been very popular recently, but providing satisfactory gaming experiences to players at a modest cost is still challenging. Colocating several games onto one server could improve server utilization. However, prior work regarding colocating games either ignores the performance interference between games or uses simple performance model to charaterize it, which may make inefficient game colocation decisions and cause QoS violations. In this article, we address the resource allocation issues for colocating games in cloud gaming. We first propose a novel machine learning-based performance model, which is able to capture the complex relationship among the performance interference, the contention features of colocated games and resource partition. Guided by the performance model, we then propose efficient and effective algorithms for two resource allocation scenarios in cloud gaming. We evaluate the proposed solutions through extensive experiments using a large number of real popular games. The results show that our performance model is able to identify whether a colocated game satisfies QoS requirement within an average error of 5 percent, which significantly outperforms the alternatives. Our resource allocation algorithms are able to increase the resource utilization by up to 60 percent compared to the state-of-the-art solutions.

Fractional Rider Deep Long Short Term Memory Network for Workload Prediction-Based Distributed Resource Allocation Using Spark in Cloud Gaming

The modern development in cloud technologies has turned the idea of cloud gaming into sensible behaviour. The cloud gaming provides an interactive gaming application, which remotely processed in a cloud system, and it streamed the scenes as video series to play through network. Therefore, cloud gaming is a capable approach, which quickly increases the cloud computing platform. Obtaining enhanced user experience in cloud gaming structure is not insignificant task because user anticipates less response delay and high quality videos. To achieve this, cloud providers need to be able to accurately predict irregular player workloads in order to schedule the necessary resources. In this paper, an effective technique, named as Fractional Rider Deep Long Short Term Memory (LSTM) network is developed for workload prediction in cloud gaming. The workload of each resource is computed based on developed Fractional Rider Deep LSTM network. Moreover, resource allocation is performed by fractional Rider-based Harmony Search Algorithm (Rider-based HSA). This Fractional Rider-based HSA is developed by combining Fractional calculus (FC), Rider optimization algorithm (ROA) and Harmony search algorithm (HSA). Moreover, the developed Fractional Rider Deep LSTM is developed by integrating FC and Rider Deep LSTM. In addition, the multi-objective parameters, namely gaming experience loss QE, Mean Opinion Score (MOS), Fairness, energy, network parameters, and predictive load are considered for efficient resource allocation and workload prediction. Additionally, the developed workload prediction model achieved better performance using various parameters, like fairness, MOS, QE, energy and delay. Hence, the developed Fractional Rider Deep LSTM model showed enhanced results with maximum fairness, MOS, QE of 0.999, 0.921, 0.999 and less energy and delay of 0.322 and 0.456.

A Skill-Based Visual Attention Model for Cloud Gaming

Despite its recent advances and increasing industrial interest, cloud gaming’s high bandwidth usage is still one of its major challenges. In this paper, we demonstrate how incorporating visual attention into cloud gaming helps to reduce bitrate without negatively affecting the player’s quality of experience. We show that current visual attention models, which work well for normal videos, underperform in the context of cloud gaming videos. Hence, we propose our novel model, by developing a skill-based visual attention model, based on a cloud gaming dataset. First, it is demonstrated how players’ attention maps are correlated with their skill levels and how this can be exploited to improve the accuracy of visual attention modeling. Then, this fact is used to cluster attention maps, according to the player’s skill level. A simple yet effective method is introduced to predict players’ skill levels using their performance in game. Finally, the models are incorporated into the video encoder to perceptually optimize the bitrate allocation. Incorporating the player’s skill level into our model improves the accuracy of saliency maps by 14% with respect to the baseline, and 24% with respect to competing methods, in terms of Normalized Scanpath Saliency (NSS). Furthermore, we show that the maximum possible amount of video bitrate reduction depends on the player’s skill level. Experimental results show 13%, 5%, and 15% reduction in video bitrate for beginner, intermediate, and expert players, respectively.

Bit Rate Reduction in Cloud Gaming Using Object Detection Technique

Bit Rate Reduction in Cloud Gaming Using Object Detection Technique

Chaotic Butterfly Optimization Algorithm for Resource Allocation in Cloud Gaming

Chaotic Butterfly Optimization Algorithm for Resource Allocation in Cloud Gaming