Benchmarking reinforcement learning algorithms in first-person shooter games using VizDoom

Reinforcement learning (RL) is a popular machine learning technique that has many successes in learning how to play classic style games. Applying RL to first person shooter (FPS) games is an interesting area of research as it has the potential to create diverse behaviors without the need to implicitly code them. This paper investigates the tabular Sarsa (λ) RL algorithm applied to a purpose built FPS game. The first part of the research investigates using RL to learn bot controllers for the tasks of navigation, item collection, and combat individually. Results showed that the RL algorithm was able to learn a satisfactory strategy for navigation control, but not to the quality of the industry standard pathfinding algorithm. The combat controller performed well against a rule-based bot, indicating promising preliminary results for using RL in FPS games. The second part of the research used pretrained RL controllers and then combined them by a number of different methods to create a more generalized bot artificial intelligence (AI). The experimental results indicated that RL can be used in a generalized way to control a combination of tasks in FPS bots such as navigation, item collection, and combat.

We present a methodology about BCI (brain computer interface) based environment system for immersion of FPS game play. FPS (First- person shooter) game is a video game genre centered on gun and projectile weapon-based combat through a first-person perspective, immersion is important factor in this game. FPS game designer effort to raise players immersion when make game using interface, graphic effect, sound etc. We implemented real-time game environment changing system using the BCI to raise the degree of immersion in FPS game. EEG 1 is the recording of electrical activity along the scalp. We measure user’s EEG using MindSet 2 for implemented game playing time to get user’s concentration rate. This value operates our environment system in virtual space. In virtual space, each player experience different environment situation because each player has various concentration rate. Implemented environment system is formed with alternation of day with night and weather. Human body condition is changed depending on day and night. Player can feel change of character’s condition via player’s condition by our system. Implemented system can be applied to various game genres to raise player’s immersion for game playing time.

Traditional single player first person shooter (FPS) games adopt a generally linear level design. Players are not given much choice as to where to go next, and thus it is paramount that the designer is able to keep the player interested throughout the whole level. It is widely accepted that in order to keep the player's interest, it is important to offer the player a varied gaming experience by presenting high interest and low interest encounters in an alternating fashion. However, while there are general theories and rules of thumb as to how this can be done, there is little formal knowledge about how exactly high or low interest levels can be achieved. Our goal is to create a better understanding on how to design encounters that affect interest levels in linear FPS games. Specifically, how exactly a player's interest levels can be raised intentionally. To accomplish this, we developed a method for measuring and comparing player interest levels based on electroencephalogram (EEG) data measured using a “Neurosky Mindset” unit, which is a commercially available EEG device. We measured player EEG data for the first 4 levels of the FPS game “Call of Duty: Modern Warfare 2” and calculated player interest levels for the entire length of gameplay. By referring to recorded gameplay videos, we were able to associate each increase in interest levels for each player to a specific point in game. From this data, we found and isolated points in each level where most players showed a rise in interest levels, and discovered that certain patterns existed between these situations. These findings led to another study where we further analyzed these situations and found out what factors caused most players to show a rise in interest levels. We were able to divide these factors into 6 different categories called Triggers: “Anticipation”, “Concentration”, “Surprise”, “Frustration”, “Overwhelm” and “Fear”. As these triggers represent mutually exclusive elements that affect a player's interest levels, we found that in most situations where many test subjects showed a rise in interest levels, more than one trigger is present, a phenomenon we call Stacking. While our study is still ongoing, we believe that by using these triggers as guidelines, game designers will be able to intentionally plan and control the player's interest level for FPS games with a certain level of guarantee.

The online FPS (First Person Shooter) games are not only widely used in entertainment, but also in serious game scenarios such as military training. In FPS games, AI is usually utilized to build NPC(Non-Player Character) such as imaginary roles of teammates or enemies. In order to provide users with more credible and intelligent AI, thereby enhancing the realism of the simulation, this paper adds affective computing to the artificial intelligence system of the online networked FPS game "Glorious Mission Online". First, it improved a hierarchical OCC affective model [1] and integrated it into the original system to drive AI to control the decision-making and behavior of the role through affective factors. Finally, the design and implementation of the prototype system in Unreal Engine are given, and actual user test data are given. The results prove that the system has a good effect on enhancing user experience.

First person shooter (FPS) games are a popular online gaming genre played predominately over wired networks. FPS games are highly interactive and have stringent delay requirements. However, the explosive growth in wireless LAN (WLAN) deployment has seen an increase in the use of such networks for gaming purposes. The varied performance of the IEEE 802.11 MAC Distributed Coordination Function (DCF) has tended to make WLANs unsuitable for FPS games. The IEEE 802.11e standard introduces quality of service (QoS) mechanisms including enhanced distributed channel access (EDCA) allowing prioritization of competing flows. Using the NS-2 simulator, we evaluate the capability of 802.11g and 802.11e WLANs to support Quake IV games traffic in the presence of web traffic. We compare the results achieved using EDCA with those achieved by a non-elevated differentiated services scheduler known as Best Effort with loss trade-off (BELT) that differentiates at the IP layer. We find that the BELT scheduler compares favorably with EDCA in this context.

Esports (also known as competitive gaming) are rapidly emerging both as an entertainment industry and in academic research. Nowadays, esports account for a large portion of gamers, viewers, and organizations. Among games played as esports, first-person shooter (FPS) games are arguably the most popular. This includes well-known titles like Quake III Arena, Counter-Strike: Global Offensive(CSGO), Overwatch, PlayerUnknown's Battlegrounds (PUBG). While these games seem to share similar game mechanics, what constitutes a popular and successful FPS game in esports is both (1) evolving rapidly and (2) have not been systematically reviewed prior. In this thesis, I qualitatively and quantitatively explore game mechanics of FPS games in esports and how they have evolved in time. This results in a taxonomy that can be used to identify specific game mechanics for different FPS games, as well as explain how these mechanics evolved over time to attract players. I conclude by discussing how the model can be used by game designers to inform both the design and analysis of FPS games and help predict future trends in game design for esports.--Author's abstract

In much of the literature the extreme distribution has been used to model first-person shooter (FPS) game packet length distributions. In this paper we show that a skewed mixture distribution, the Ex-Gaussian, is also suitable for modelling the packet payload lengths for two-player games of seven popular FPS games in the server-to-client direction. Also there is a plausible physical justification for the choice of a mixture distribution as a suitable model. The Ex-Gaussian distribution has properties that can be exploited to synthesise the server-to-client packet payload length distributions for larger numbers of players using measurements taken from game trials with a small number of players. We have also outlined a computationally simple technique that can be used to synthesise FPS game server-to-client packet payload length distributions for N-players from measurements from two-player games. This technique is useful for building realistic traffic models for FPS game traffic that can be used in simulation studies.

P2P consistency support for large-scale interactive applications

Video games present a unique opportunity to study motor skill. First-person shooter (FPS) games have particular utility because they require visually guided hand movements that are similar to widely studied planar reaching tasks. However, there is a need to ensure the tasks are equivalent if FPS games are to yield their potential as a powerful scientific tool for investigating sensorimotor control. Specifically, research is needed to ensure that differences in visual feedback of a movement do not affect motor learning between the two contexts. In traditional tasks, a movement will translate a cursor across a static background, whereas FPS games use movements to pan and tilt the view of the environment. To this end, we designed an online experiment where participants used their mouse or trackpad to shoot targets in both visual contexts. Kinematic analysis showed player movements were nearly identical between contexts, with highly correlated spatial and temporal metrics. This similarity suggests a shared internal model based on comparing predicted and observed displacement vectors rather than primary sensory feedback. A second experiment, modeled on FPS-style aim-trainer games, found movements exhibited classic invariant features described within the sensorimotor literature. We found the spatial metrics tested were significant predictors of overall task performance. More broadly, these results show that FPS games offer a novel, engaging, and compelling environment to study sensorimotor skill, providing the same precise kinematic metrics as traditional planar reaching tasks.

The main purpose of this study is to develop an useful technique for the military fire training and first person shooting (FPS) game, and obtains the solution of the man-machine interface for its interactive operations. This study use a laser guide technology for guiding an aiming point of fire, and the control function includes the position and direction information. For controlling the aiming point of fire, this paper introduces a laser emitter mounted on the muzzle, and utilizes an optical camera with a light filter lens as a receiver to detect the laser light pointed on the screen. After some image process and computer software program operations, the movement of image can be transformed by a cursor command to guide the aiming point of fire which includes the position and direction control function. On the other hand, we also apply the wireless control technology that is an effective method to make an instruction to initiate the computer for executing the fundamental operations mode. To combine laser guide and wireless control multi-technology on simulation shooting gun, this innovation design demonstrates excellent results on the simulation of live fire shooting and extreme suits in various types of FPS game.

The current study aimed to understand the effects of playing video games on navigation skills, spatial cognitive abilities and speed of processing, and to examine the characteristics of games and training hours impacting video game training. Participants were assigned to one of four groups; First Person Shooter (FPS) game training group, Third Person Shooter (TPS) game training group, Control puzzle game training group, and control non-training group. Participants in training groups played the selected games for 30 hours in total. Performances in maze tasks, spatial attention, mental rotation and speed of processing were measured at four time points (pretraining, post-10 hours, 20 hours and 30 hours training) to investigate improvements over the time. The results showed the FPS game playing and not the TPS or puzzle game playing enhanced visual attention ability at both 20° and 30° eccentricity. This result indicates viewpoints of video game are important characteristics which may impact differently on cognitive ability. However, no significant improvement was found in navigation skills, suggesting playing FPS game may not improve dynamic and larger-scaled spatial abilities beyond spatial attention.

Traditional client-server online games do not scale well in terms of the number of players they can support. Most of this is due to the quadratic growth of bandwidth requirements as the number of players increases, and the limitations on processing power of any single machine. Considering the excitement a first person shooter (FPS) game can provide by bringing an epic scale online battle to real life, we present a scalable cloud-based architecture able to host hundreds of players in an online FPS game. We host the game in a cloud, rather than on a single machine, and reduce aggregate bandwidth requirements of the game by using a scalable publish-subscribe subsystem. Each player expresses its preferences about other players using an interest set, and receives updates accordingly. Our evaluations, both in a testbed and cloud environment, show our architecture can scale to hundreds of players, an order of magnitude more players than state-of-the-art FPS game servers can currently support.

Gender, native English speaking, and age significantly effect game genre preference and gaming motivations. Ordinary Least Squares (OLS) regression shows they explain 5 %–10 % of the variance in game genre preference and up to 7 % in gaming motivation. Gender coefficients show males prefer the competition-based First Person Shooter (FPS) games while females prefer the immersion-based Massively Multiplayer Online Role Playing Games (MMORPG). Native English speaking coefficients show native English speakers prefer the text-heavy MMORPGs, while non-Native English speakers prefer the text-light Multiplayer Online Battle Arenas (MOBA) and FPS games. Age coefficients show younger players prefer MOBAs, while older players prefer FPS games. When it comes to gaming motivation, males are more driven by competition, while females are more driven by immersion and social motivations. Native English speaking only factors into two motivations related to immersion. Age coefficients show that gaming motivation decreases across the board as players grow older.

The amount of interest in Virtual Reality (VR) research has significantly increased over the past few years, both in academia and industry. The release of commercial VR Head-Mounted Displays (HMDs) has been a major contributing factor. However, there is still much to be learned, especially how views and input techniques, as well as their interaction, affect the VR experience. There is little work done on First-Person Shooter (FPS) games in VR, and those few studies have focused on a single aspect of VR FPS. They either focused on the view, e.g., comparing VR to a typical 2D display or on the controller types. To the best of our knowledge, there are no studies investigating variations of 2D/3D views in HMDs, controller types, and their interactions. As such, it is challenging to distinguish findings related to the controller type from those related to the view. If a study does not control for the input method and finds that 2D displays lead to higher performance than VR, we cannot generalize the results because of the confounding variables. To understand their interaction, we propose to analyze in more depth, whether it is the view (2D vs. 3D) or the way it is controlled that gives the platforms their respective advantages. To study the effects of the 2D/3D views, we created a 2D visual technique, PlaneFrame, that was applied inside the VR headset. Our results show that the controller type can have a significant positive impact on performance, immersion, and simulator sickness when associated with a 2D view. They further our understanding of the interactions that controllers and views have and demonstrate that comparisons are highly dependent on how both factors go together. Further, through a series of three experiments, we developed a technique that can lead to a substantial performance, a good level of immersion, and can minimize the level of simulator sickness.

While target selection in a 2D space is fairly well-studied, target selection in a 3D space, such as shooting in first-person shooter (FPS) games, is not, nor are the benefits to players for many latency compensation techniques. This paper presents results from a user study that evaluates the impact of latency and latency compensation techniques on 3D target selection via a bespoke FPS shooter. Analysis of the results shows latency degrades player performance (time to select/shoot a target), with subjective opinions on Quality of Experience (QOE) following suit. Individual latency compensation techniques cannot fully overcome the effects of latency but combined techniques can, letting players perform and feel as if there is no network latency. We derive a basic analytic model for the distribution of the player selection times which can be part of a simulation of a full-range of FPS games.